In recent years, methods from machine learning have make great advances to solve difficult problems in artificial intelligence. In particular, (deep) feed-forward neural networks have re-established themselves as one of the most powerful learning architectures. This has led to spectacular applications in the area of computer vision, speech recognition and game playing. The current successes are pushing the technology to larger and larger applications, pushing the limits of computation and energy consumption. The learning algorithms for deep networks have much improved over the last years along various directions. However, good performance is still hard to achieve because of unreliable gradients due to sampling difficulty and large mixing times in stochastic networks. For recurrent neural networks, there are no good algorithms for learning general (no detailed balance) stochastic networks. Furthermore, there is no good algorithms to learn with low precision synapses. In this proposal, we aimto advance algorithms and theoretical understanding of learning in two directions. We study learning with low precision stochastic synapses by formulating the neural networks as Markov processes and treat the learning as a stochastic optimal control problem. We analyze the properties of these networks using replica analysis. Secondly, we aim to improve computational efficiency by formulating quantum mechanical equivalents of learning. The learning rules from this project aim to provide the algorithms for on-chip learning on future neuro-morphic hardware devices. An important limitation for the growth of future generations of computers, and thus of neural networks, is the excessive amount of heat that they produce. Devices that allow for on-chip learning and that replace double precision synapses with unreliable stochastic bits, can potentially save many orders of magnitude in energy consumption and heat dissipation and thus scale-up neural network computation. The stochastic approach to learning proposed in this project naturally yields learning rules that are more local than traditional back-propagation based learning, and are promising candidates for on chip learning.
We seek an excellent and highly motivated PhD candidate with a MSc degree in theoretical physics and good computational skills. We offer a full time research position for 4 years. Your salary will be up to a maximum of 2834 euro gross per month. The salary is supplemented with a holiday allowance of 8 percent and an end-of-year bonus of 8.33 percent. You are supposed to have a thesis finished at the end of your four year term. The starting date of the project is July 1, but can be somewhat delayed if needed.
The project will be executed at the SNN Machine Learning research group of Bert Kappen. www.snn.ru.nland www.snn.ru.nl/~bertk. The project is part of a joint project with Riccardo Zecchina of the University of Milan. Frequent contacts to and from Milan are expected.
The candidate has a MSC degree in theoretical physics and good computational skills.
Conditions of employment
When fulfilling a PhD position at the NWO Institutes Organisation (NWO-I), you will get the status of junior scientist.
You will have an employee status and can participate in all the employee benefits NWO-I offers. You will get a contract for four years. Your salary will be up to a maximum of 2,834 euro gross per month.The salary is supplemented with a holiday allowance of 8 percent and an end-of-year bonus of 8.33 percent.
You are supposed to have a thesis finished at the end of your four year term with NWO-I.
A training programme is part of the agreement. You and your supervisor will make up a plan for the additional education and supervising that you specifically need. This plan also defines which teaching activities you will be responsible (up to a maximum of ten percent of your time). The conditions of employment of NWO-I are laid down in the Collective Labour Agreement for Research Centres (Cao-Onderzoekinstellingen), more exclusive information is available at this website under Personeelsinformatie (in Dutch) or under Personnel (in English).
General information about working at NWO-I can be found in the English part of this website under Personnel. The 'Job interview code' applies to this position.
Prof. H.J. Kappen
15 Augustus 2017