I Build The Black Box: Grappling with Product and Policy:
The rate of improvement in techniques for building machine learning models over the past 2 years has been astounding; between generalized embedding models like starspace and scalable, portable classifiers like XGBoost now mean that we can compress months of work into days or even hours. Unfortunately, we have not had any similar improvements in our ability to solve the product and policy problems that so often go hand-in-hand with building and deploying models; if anything, our reliance on self-optimizing black box techniques means that these problems are only getting harder, and as we bring machine learning to bear on more diverse domains, the stakes are only getting higher.
Bio: Josh Wills is the head of data engineering at Slack. Prior to Slack, he built and led data science teams at Cloudera and Google. He is the founder of the Apache Crunch project, co-authored an O’Reilly book on advanced analytics with Apache Spark, and wrote a popular tweet about data scientists.