Dunning - SIGMOD - Data Economy.pptx
•0 likes•19 views
Download to read offline
Report
We introduce the idea that metadata, including project information, data labels, data characteristics and indications of valuable use, can be propagated through a data processing lineage graph. Further, finding examples of significant cooccurrence of propagated and original metadata gives us the basis of an interesting kind of search engine gives interesting recommendations of data given a problem statement even in a near cold-start situation.
Recommended
More Related Content
Similar to Dunning - SIGMOD - Data Economy.pptx
Similar to Dunning - SIGMOD - Data Economy.pptx(20)
More from Ted Dunning
More from Ted Dunning(20)
Recently uploaded
Recently uploaded(20)
Dunning - SIGMOD - Data Economy.pptx
- 1. FROM ROOTS TO FRUITS: EXPLORING LINEAGE FOR DATASET RECOMMENDATIONS Ted Dunning, Fellow, HPE 18 June, 2023
- 2. the meaning of words lies in their use 2
- 3. the meaning of words lies in their use 3 the meaning of data lies in its use (apologies to Dr. Wittgenstein)
- 4. 4 A meteorologist’s data - rainfall - windspeed - temperature
- 5. 5 A meteorologist’s data - rainfall - windspeed - temperature A business uses the data to predict umbrella sales
- 6. 6 What does the data actually mean?
- 7. 7 What does the data actually mean? the meaning of data lies in its use
- 8. TRAINING PROCESS 8 README URL History Datasets + Models Metadata We start with explicit metadata. Examples: column and table names, documentation, common values, and others
- 9. TRAINING PROCESS 9 README URL History Datasets + Models Metadata This is encoded as a large artifact x characters incidence table At this point, direct metadata search is possible
- 10. TRAINING PROCESS 10 README URL History Datasets + Models Metadata We augment with metadata from all ancestors and descendants in the global data lineage graph
- 11. TRAINING PROCESS 11 README URL History Datasets + Models Metadata Finally, we reduce the characteristic cooccurrences using indicator-based recommendation methods.
- 12. A NOTE ON IMPLICATIONS 12 The characteristic indicator matrix is what connects “umbrella” with “rainfall” or “mosquito” with “temperature” + “windspeed”
- 13. QUERY PROCESS 13 The original query is often textual, possibly a README
- 14. QUERY PROCESS 14 augmented by recent project behavior (queries, references)
- 15. QUERY PROCESS 15 The query is expanded based on indicators (when they say “umbrellas” they also mean “rainfall”) as well as semantic token embedding using BERT
- 16. Recommendations Explanation positives.csv notpositives.csv SARIMA_model dengue_monthly.csv climate_monthly.csv “dengue” ancestor “dengue” ancestor “dengue” ancestor “dengue" “wind speed” QUERY PROCESS 16 The final results include an explanation of why files or programs are included.
- 17. Recommendations Explanation positives.csv notpositives.csv SARIMA_model dengue_monthly.csv climate_monthly.csv “dengue” ancestor “dengue” ancestor “dengue” ancestor “dengue" “wind speed” QUERY PROCESS 17
- 18. EVALUATION • Evaluation is difficult due to a lack of public datasets • Most machine learning examples are truncated to final steps • Very few non-machine learning pipelines exist outside of toy examples • Private datasets generally cannot be shared • Still important to use when possible due to scale • Evaluation of recommendation engines is a subtle art • Their purpose is to change behaviors • Todays recommendations select tomorrow’s training data • We aren’t to this point yet, this would be a symptom of success 18
- 19. EVALUATION 19
- 20. EVALUATION 20