The Von Neumann architecture is characterized by its serial processing nature, fetch-decode-execute cycle, and use of a common bus. It has four main components: a processor, memory, I/O devices, and secondary storage. The processor contains an arithmetic logic unit and control unit, and uses registers like the program counter, memory address register, and accumulator. Instructions are fetched from memory one at a time by the processor and executed sequentially, limiting processor speed.