A distributed computing system is a collection of interconnected processors with local memory that communicate via message passing. There are various models including minicomputer, workstation, workstation-server, and processor pool. Distributed systems provide advantages like supporting distributed applications, sharing information and resources, extensibility, shorter response times, higher reliability, flexibility, and better price-performance ratio compared to centralized systems.