The World Wide Web (Web) is a visually complex, dynamic, multimedia system that can be inaccessible to people with visual impairments. SADIe uses semantic annotations of a Website's Cascading Style Sheet (CSS) to drive a transformation process that can improve access to content for visually impaired users. The original process of annotating the CSS involved the use of an upper ontology, extended by a site specific lower ontology. While this approach provided rich annotation of the CSS terms, experience suggests that components within the model were inappropriate for the interactive system we were developing. This experience has led to a more pragmatic approach that still provides the necessary semantics required to drive the SADIe transcoding tool, but in a more lightweight manner. This paper describes the lessons learnt from building the ontological models for the SADIe platform, highlighting pitfalls that developers of ontologies in interactive systems should be wary of.