Container driven development, build pipelines and technology

Hi all,

I’ve been working with a newer technology stack of late, involving using Docker and Rancher to host my services, having come from a Microsoft tech background, I’ve found this to be quite the shock, but it does feel like finally, the tools that are out there to support integrating, maintaining and releasing different technologies are starting to catch up.

I’ve always been a bit frustrated by the amount of time it takes to take open source technologies and use them across the full pipeline, because it can take a lot of effort to get everything working harmoniously together. There are conflicting opinions on the best architectures, tools and techniques to use, and often standards are open to debate too, making it difficult to reach a point where you are satisfied with what you’ve got. Coming from a .NET, TFS and IIS background, I never struggled with this since most of the integration work was done for me, standards were readily available from the vendor, and while I may have disagreed with them, or they made my life difficult at times, mostly they worked well and there wasn’t another option anyway, meaning I never really spent much time looking for alternatives.

Working with AngularJS, Node, Docker, Jenkins, Quay and Rancher has been tricky, and when I started looking at container based development, I found it irritating that most of the work was extremely manual, I had to plug a lot of gaps to get these things working that I never worried about before and it required me to recall several commands with various switches and syntax in order to get my code working, via command line interfaces and shell scripts, something quite alien to me.

Instead of worrying about how my system worked, and what it needed to do, I spent a lot of time just trying to get it to work at all, and then working out the various ways to build a pipeline. As Jakob Nielsen has stated in system design over the years, you want to strive for recognition over recall, because if I’m not in that command line every day and using those commands, maintaining those shell scripts I’m not going to remember them, I need tooling that allows me to recognise the right sequence of events visually or through a workflow, because when I am putting together this complicated JigSaw, only me and God know what’s going on, and in a years time after not touching them, God only knows.

I don’t want have to pour over Stack Overflow answers to workout which switch I’m missing in a docker command or make calls to API’s from a shell script, it seems counter productive to me to have to spend so much time doing that when I’ve got so many other technologies and frameworks making up my system and pipeline, it’s got to be made easier, there is too much time between situations where I need to do it, so anything I write in this way is rusty┬áby the time I need to do it again.

All of this may give the impression I’m against this new way of development and deployment, in fact, I fully support it, but what I find frustrating is that within a corporate environment with market pressures and delivery windows to hit in order to keep a competitive advantage, the current tools that are available to work with these technologies are too complex and time consuming, and finding developers who understand how to use them and want to is so difficult outside of London. The tools are finally here to support this development and to give us the layers of abstraction we want, but they need refining, it feels like we’ve discovered the hand saw and we can finally craft our systems in the ways we would want, but what I really want now is a table saw to speed up the process.