This paper introduces techniques for Deep Learning in conjunction with spiked random neural networks that closely resemble the stochastic behaviour of biological neurons in mammalian brains. The paper introduces clusters of such random neural networks and obtains the characteristics of their collective behaviour. Combining this model with previous work on extreme learning machines, we develop multilayer architectures which structure Deep Learning Architectures a a “front end” of one or two layers of random neural networks, followed by an extreme learning machine. The approach is evaluated on a standard - and large - visual character recognition database, showing that the proposed approach can attain and exceed the performance of techniques that were previously reported in the literature.