Be the first to like this
SUSTAINABILITY SUPPORTING DATA VARIABILITY: KEEPING CORE COMPONENTS CLEAN WHILE DEALING WITH DATA VARIABILITY
A big challenge in building complex, data-intensive systems is how to sustainably support data variation, schema, and feature evolution. This talk examines strategies, practices, and patterns drawn from real experiences that support new and evolving data-processing requirements while keeping the core architecture clean. As complex systems evolve to meet varying data formats, they can devolve into poorly architected Big Balls of Mud filled with special-case logic and one-off processing. Alternatively, you can isolate core components of your system and protect them from entanglements and unnecessary complexity by designing them to operate on common data formats while providing extension mechanisms that enable processing variations.