DistCI (https://github.com/F-Secure/distci) is an attempt at building distributed, horizontally scalable and fault tolerant continuous integration system. It is aimed to have the capability to manage thousands of jobs and hundreds of concurrent tasks with no single points of failures or bottlenecks. DistCI utilizes CEPH for distributed data storage and ZooKeeper for global synchronization. Presentation will cover challenges with the existing CI systems, design principles behind DistCI, and future outlook. Key concepts include worker led coordination, chunking of build tasks into stackable flows and distribution of the tasks over large number of nodes.
Target audience for my presentation is developers interested in ubiquitous test automation. If you demand proof via TA on each and every commit before acceptance for merge, DistCI might be of your interest.