escience2019 has ended
Back To Schedule
Wednesday, September 25 • 4:45pm - 5:15pm
Out-of-the-box Reproducibility: A Survey of Machine Learning Platforms

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Richard Juul Isdahl (Norwegian University of Science and Technology) and Odd Erik Gundersen (Norwegian University of Science and Technology)

Even machine learning experiments, which are fully conducted on computers are not necessarily reproducible. An increasing number of both open source and commercial machine learning platforms are being developed that help address this problem. However, there is no standard for assessing and comparing which features are required to fully support reproducibility. We propose a quantitative method that alleviates this problem. Based on the proposed method we assess and compare the current state of the art machine learning platforms for how well they support making empirical results reproducible. Our results show that BEAT and Floydhub have the best support for reproducibility with Codalab and Kaggle as close contenders. The most commonly used machine learning platforms provided by the big tech companies have poor support for reproducibility.


Odd Erik Gundersen

Norwegian University of Science and Technology

Wednesday September 25, 2019 4:45pm - 5:15pm PDT
Macaw Room