{"id":2260,"date":"2016-08-09T01:23:09","date_gmt":"2016-08-09T00:23:09","guid":{"rendered":"http:\/\/www.xenonique.co.uk\/blog\/?p=2260"},"modified":"2019-12-03T18:14:18","modified_gmt":"2019-12-03T17:14:18","slug":"ye-olde-monolith","status":"publish","type":"post","link":"https:\/\/www.xenonique.co.uk\/blog\/2016\/08\/09\/ye-olde-monolith\/","title":{"rendered":"Ye Olde Monolith"},"content":{"rendered":"

As a professional software engineer, you probably have encountered an architectural stumbling block known as a MONOLITH<\/a>#1<\/span><\/a>. A monolith is essentially a single tier application in enterprise architecture. Forget about the traditional break-down of Java EE application, where they break down an Uber-WAR<\/a> file into presentation, business-logic and database tiers. That is just gloss over our very eyes, a single WAR file deployed to an application server is, at best, a monolith.<\/p>\n

\n
Monolithic reference<\/div>\n
\nThe reference probably comes from the movie 2001 A Space Odyssey<\/a>, which describes the monolith as a machine designed by an extraterrestrial alien species. The term is probably derived from Arthur C. Clarke\u2019s Space Odyssey novels and films. In the author’s stories, monoliths are influential in shifting evolution on Earth and in the software architecture narrative a monolith is a huge inertial aggregation of components that are extremely difficult to evolve.\n<\/div>\n<\/div>\n

<\/p>\n

In my contracting experience, I have witnessed many types of monoliths at various stages of software development lifecycle. Unfortunately, I have been towards to the latter end of SDLC<\/a> where the monolith application is already in maintenance mode or has been reiterated several times. It is the nature of contracting that the conditions of bit rot have already been set and they are irreversible.<\/p>\n

A monolith has the following characteristics:<\/p>\n

    \n
  1. The application code is generally unfathomable, because the modules are inextricably interlinked making analysis absurdly complex.\n<\/li>\n
  2. The time of development cycles gradually increase through each milestone release.<\/li>\n
  3. There is a vast swathe of code and modules. I would say over 500,000 lines of code and over 250 modules is de-rigour.<\/li>\n
  4. The ratio of unit test classes to regular code is alarmingly very low, sometimes much less than 5%.<\/li>\n
  5. Usually there are only one or two persons in the organisation who has any clue how the whole monolith hangs together.<\/li>\n
  6. There is a culture of resistance to change. Sometimes that culture is political.<\/li>\n
  7. People have passed through the eye of the needle many times over.<\/li>\n<\/ol>\n

    Initialisation<\/h2>\n

    How do MONOLITHS come to being? Who is responsible? I have my suspicions about the origin. In my experience, they are started when a business generates and builds software from a brownfield #2<\/span><\/a> environment. <\/p>\n

    \n
    Does a greenfield ever exist?<\/div>\n
    \nRarely have I ever seen Greenfield development projects<\/a> that look like they will eventually become monoliths. For one thing, it is difficult to know how an application will turn, once you have finished the consultancy engagement, left the company for a new one and the client has 6 months to go to reach the first release candidate milestone. Once a project has started it with the baby prototype, basic CRUD there is strong resistance from the initial development to throw out the code and start afresh. The concept of the original
    \nseems to hang on in all shape and all form.\n<\/div>\n<\/div>\n

    <\/p>\n

    How does a monolith actually start? It starts with basic requirement or that appears to be fundamental and then deviates into the asylum from the beginning date. The weird thing is that it does not matter if you have average engineers or a crack team of skunk workers, with at least 10 years\u2019 experience and, say, that each person had worked as technical architectural leaders in the past. Software, unfortunately, is very subjective. We have all opinions on the best framework, the best library and we are limited by the constraints we have in our knowledge at the time of the monolith’s conception. <\/p>\n

    Why do I call the initial conditions brownfield<\/a>? Because there is rarely a case in a prototype or stareter version where we build everything from scratch. A monolith starts with a dependency like Java EE, if the code is written in Java. It will be different if your monolith is written in PHP, Python, Groovy, Scala or Ruby. <\/p>\n

    In the era before Agile, the monolith came to being through a Big Upfront-Design<\/a> (BUFD), Project Initiation Document<\/a> (PID) and New Enterprise Architecture Document <\/a>(NEAD). <\/p>\n

    Here is an extract of old proposal document that I have:<\/p>\n

    \nUniversal Trade Desk Add-On<\/p>\n

    Executive Summary:
    \nWhat is UTD and why are we building it?
    \nHow is it being implemented?
    \nWhen will it be delivered and what will be included?
    \nWhere are we now?
    \nSummary and future development \/ roadmap<\/p>\n

    UTD Objective<\/p>\n

    \u201cThis new application will be a near real-time Trading Desk Add-On to provide consolidated single source of Trade Data in a format understandable by X-QuantLib Exposure Library (X-QEL). The contents of UTD will be updated upon trade lifecycle events and will be available via an API. Traders will be migrated from LEGACY-Y to UTD in progressive steps. UTD will be available across the Web and also Desktop profiles.\u201d\n<\/p><\/div>\n

    <\/p>\n

    From the above sample description, which was written in 2009, I have changed some terms in order to protect the innocent, you can see exacting business knowledge [jargon] is required#5<\/span><\/a>. If I could have returned to this monolith in 2016 and re-examined this application, it would likely be still a monolith, and I doubt that I would find the original documentation that described the start of the journey. In fact, I have engaged in a few contracts and the client cannot longer lend their hands on the original BUFD.<\/p>\n

    \n
    Domain-Driven Design<\/div>\n
    Domain-driven designers will know that is the key focal point for creating the ubiquitous language<\/a> between technical and business people. DDD emphasises the process of developing communication between the ultimate users of the software and the people responsible for developing it.\n<\/div>\n<\/div>\n

    <\/p>\n

    \n\"architecture-monolith-box\"\n<\/div>\n
    \nFigure 1: The sad story of the Ye Olde Monolith.\n<\/div>\n
    \n

    Throwaway software and lost versions<\/h3>\n

    A long while ago, when I was learning to program computer games at secondary school, we shared a simple school computer that used floppy disks for storage. I learnt the BASIC programming language<\/a>. One of the commands that I knew was SAVE A:GAMEFILE.BAS<\/code> that saves current state of the program code to a file, where A:<\/code> represents the location of the floppy disk device. One day, I got so far into my day\u2019s programming in between lessons in maths, English, physics, chemistry, geography, history, general studies etc.; I forgot to save the program at regular intervals. I was so excited and in the flow of the interactive that I can\u2019t remember what happened, maybe the PC crashed or I ran out to the next lesson or something else. In any case, I lost my entire work; I was really annoyed about it! I eventually retyped everything from my memory, but here\u2019s the thing, I figured out how to do it better. Second time around I considerably improved the game with better subroutines and new concepts. I knew about Frederick Brook’s long before I had read the book!\n<\/div>\n

    <\/p>\n

    Culture<\/h2>\n

    A monolith is characterised by the culture that creates it. As soon as code is written, some say, you are already in a technical debt. In effect, writing code even with unit tests is like taking an option on the future outcome. A fully unit test harness is the guarantee reference and a promise to yourself that you might change the production code eventually. Moreover, if you know that code is just a prototype then you can afford to cut corners and deliver the fastest working example of the application that impresses the stakeholder.<\/p>\n

    The prototype that eventually grows into the ageing monolith rarely ever gets thrown away or restarted. There is an early day scenario where the majority of the application should be rebuilt from scratch, but because of business pressures like cost, time-to-market and finance, nobody ever does this. Over time the monolith keeps growing and stretching further into the distance. <\/p>\n

    After several Person-years this mountain of code keeps growing every day, every month and every year. Why? Because a monolith is nothing except a transformation of a viable business model, in other words as long the business can makes profit [or it thinks that it can] then it will self-sustain#7<\/span><\/a>. <\/p>\n

    \n
    Tenuous business model<\/div>\n
    \nTo be at affect of crumbing monolithic application, is probably the worse position to find yourself in, especially if it is your first day at work as Global Head of Development. One would hope that actually you have been blessed organisational director powers, motive and belief that you can change the scene.\n<\/div>\n<\/div>\n

    <\/p>\n

    Monoliths to date are also a victim of software engineering trends. We can look at the technology and the libraries. The oldest Java monoliths, probably, were created long before the creation of Spring Framework, Guice and Context Dependency Injection. Some monoliths, therefore, were conceived when Java did not have annotations or even generics, which is pretty old. Even the best practice of the times change with time. By 2016, the J2EE design patterns<\/a> are woefully past their sell-by date and I would also add action controllers written against Apache Struts version 1.0 to the mix.<\/p>\n

    At best, the monoliths were never modularised or designed for best practice of components#3<\/span><\/a>. Joe Nuxoll was one of the first people that ever heard who was pushing componentisation design and properties for software<\/a>, unfortunately, Java the language never did get there.<\/p>\n

    \n
    What will you do when Java 9 hits?<\/div>\n
    \nThose of you might be stuck in the dirt when it comes to Java 9. Many engineers see Project Jigsaw as a terrific opportunity to remove legacy components from the JDK and their applications. Of course, it remains to be seen, and architects know that their applications would already exhibit great cohesion and severely reduced coupling in order to benefit from modularisation. By the way, I probably would bail on a JDK 1.6 based monolith project if I were you and get straight onto a new contemporary project as soon as…Run!<\/strong>\n<\/div>\n<\/div>\n

    <\/p>\n

    Conway\u2019s Law<\/h3>\n

    Conway\u2019s Law<\/a> asserts that organisations are constrained to produce application designs that mirror their communication infrastructures. No matter how adaptive an organisation tries to break out beyond the law, the result always is unintended communication breakdowns between management, departments and silos; which all require resolution. <\/p>\n

    Unsurprisingly, monoliths will exhibit the structure of the people that created and also maintain them. <\/p>\n

    \n\"software-entropy-cost-6-months\"\n<\/div>\n
    \nFigure 2: In the beginning, the excitement is palpable, the moral high, every developer believes they are part of a crack team that will win the World Cup! Software entropy is very deceptive, because it never ever decreases!\n<\/div>\n

    Staff turnover<\/h3>\n

    A monolith is also affected by staff turnover<\/a>. If new business team has an original idea of an application at the very beginning, after a few years the application’s direction will change. Because existing staff leave the business, move on to new projects and learning; and also original consultants and contractors leave the project, the arrow of development changes orientation as new people arrive. The new staff must learn new concepts and they rarely handed over in fashion that shows maximum knowledge transfer. <\/p>\n

    \nI once went down to an interview in London, several years ago. I had no idea of the situation before sitting down and discussing the contract with the hiring manager. The prospective client had a major issue with legacy Java code. They wanted to hire a seasoned Java enterprise engineer to look at and maintain it. The Java platform was on its last legs. They also informed me that internal staff and modern contractor had moved their platform to Scala and that is the language they were using for, then, current development. Of course, I was slightly amused but this. I asked how many lines of Java code was there? They didn\u2019t know exactly and estimated 750,000 lines of code. I didn\u2019t consider this opportunity, one guy against three quarter of millions lines of code, whilst the new kiddies delightfully worked their heart out on Scala. They could blow that \u2026 and they did.\n<\/div>\n

    <\/p>\n

    Undersell<\/h3>\n

    I came across at least two monoliths in my time, which were the product of software houses. The monoliths were large and at least one of them was supremely huge by modern standards, because they factored in JavaScript and modern front-end libraries to solve the Single Page Application<\/a> paradigm #4<\/span><\/a>. <\/p>\n

    \n
    Single page architecture!?!<\/div>\n
    \nI will have more to say about monolith and single page applications in a subsequent article.
    \nPlease stay tuned!\n<\/div>\n<\/div>\n

    <\/p>\n

    Because the business relied on selling their software to customers as a bespoke monolith, there were most likely promises to get extensions to the monolith finished by a certain date. I suspect that the extension and final application were underestimated and like the majority of software projects in the industry finally arrived considerably over the expected budget. <\/p>\n

    The problem is that monoliths have within themselves undue complexity in terms of logic. An extension of the application is dependent on several factors:<\/p>\n

      \n
    1. Comprehensive view of the monolith\u2019s internal working components\n<\/li>\n
    2. Sufficient translation of new business requirements to technical requirements\n<\/li>\n
    3. Adapting old components and developing new components\n<\/li>\n
    4. Dealing with the stress of mixed capable staff\n<\/li>\n
    5. Unknown impedances in the code, framework and databases\n<\/li>\n<\/ol>\n

      These above are the forces acting on an ageing monolith. At the very beginning of the SDLC, these forces are still there, however the complexity of the code and initial requirements are smaller.<\/p>\n

      Time to market<\/h2>\n

      Monoliths suffer because of management insistence on delivery to a fixed budget. Of course, it means that due-diligence on the code never takes place and therefore the overall production quality suffers. Developers don’t have time to refactor the code to sufficient quality, code reviews dry up. Morale is destroyed by having to be spoon fed from the political power person during pair-development. It takes longer and longer to write a simple change like adding new field to a HTML form, because architecturally it means trawling though front-end JavaScript framework and altering AJAX, bundling JSON; finding and adapting server-side controllers, changing any database persistent objects and then adding yet another a slap-booty XML configuration to the multitude of LiquiBase scripts<\/a>. Engineers lose Person-days as you can imagine, because the monolith is so far removed from the Java EE CRUD example that you see in a beginner\u2019s book, and then management wonder what the development team are really archiving? This is the complexity budget, which is sadly, overlooked by management.<\/p>\n

      Lack of unit tests<\/h3>\n

      Depending of the age of the monolith they demonstrate a shocking minimal degree of unit test coverage. The worst monolith that I ever saw had a ratio of 0.2% test coverage according to SonarQube<\/a> analysis #6<\/span><\/a>. <\/p>\n

      The lack of unit tests revealed that obviously time-to-market was a crucial factor in past. Some application monoliths were written before even dependency injection had taken a foot hole in the industry. There were so many static class factory singletons that it made testing incredibly difficult. Although, there was one client that I saw had various degree of success with Power Mock<\/a>. Even with those monolith that had unit tests, invariably they were not verifying or validating enough outputs against expected inputs. The test were brittle and weak. In some cases, the tests had been deliberately removed and commented out, because I could inspect the version control history log!<\/em><\/p>\n

      \n
      Interpretation or intelligence<\/div>\n
      \nThe internal staff had glibly decided to ignore the statistics.\n<\/div>\n<\/div>\n

      <\/p>\n

      Lack of know-how<\/h3>\n

      With dependency injection and modules in general, the industry has shown over the past decade a lack of genuine know-how. It takes a while to design a modular application system effectively and most people need several year’s experience. <\/p>\n

      We have searched for short cuts in building systems, because of cost-pressure, time-pressure and, of course, monetary-pressure. We have struggled over 25 years attempting to build Object-Oriented applications and systems. It is hardly a surprise that monoliths come into being, because there is plenty of evidence of what not to do: anti-patterns<\/a>. <\/p>\n

      Failure to keep up-to-date<\/h3>\n

      As we add more features and functions to a monolithic application and in particular one that we never throwaway, we end up layering complexity on more complexity. It is a bit like having an open wound. The sensible thing would be put an Elastoplast<\/a> on it, because we think it is the quickest way to alleviate the pain and suffering. However an open wound is not a straightforward cut. It requires antiseptic, cleaning, minor, or sometimes major surgery, and then proper bandages followed by full time it takes for recovery. Software engineers hardly ever get the facilities and support to clean up bad code or change the code smells. It stay infected and becomes worse. <\/p>\n

      Meanwhile the software development continually finds a brand new way. The Java language, then, introduces annotations in one year. In the following years it introduces functional interfaces, Lambdas and Streams. Yet with a huge mass of monolith with severe complexity and incomprehensibility, it is nearly impossible to use any of those recent features. For one, the management or the stakeholders that fund the monolith and keep the good ship afloat at ship are petrified of the risk of any advancement. For two, politically and often the business model means that they deliberate fail to keep the monolith up to date with technology. And then there is the consideration for Java 9 and Project Jigsaw.<\/p>\n

      \n\"software-entropy-cost-6-years\"\n<\/div>\n
      \nFigure 3: Over the software development lifecycle of this monolith, which is now in maintenance after 6 or 7 years of progress, we can see the complexity is causing issues in technical and the business axes.\n<\/div>\n

      Can a monolith be beautiful?<\/h2>\n

      Recently, I listened to episode 261 of Software Engineering Radio<\/a> featuring the very opinionated David Heinemeier-Hanson<\/a> of Ruby on Rails<\/a> and Basecamp <\/a>fame. <\/p>\n

      DHH describes his views on the hype about micro-services oriented architecture (M\/SOA)<\/a>. He thought that we were blinded and prone to over-engineer<\/em> new systems as micro services. Instead, he pointedly described Basecamp as a \u201cmajestic monolith\u201d<\/em> that benefited from a dynamic programming language; and over the past decade optimisations in operating system functionality and CPU performance. DHH argued against going for distributed computing application at the outset, especially in a small setup. <\/p>\n

      I found his views reassuring for the monolithic application. However, I observed that Basecamp, and I have never seen the code, has been under his complete control since the beginning. Every check-in of code into the version control system was most likely validated. DHH and Jason Fried<\/a> are unusual, because they are the architects who have stayed with their software. Technical software architects often drop out of coding activities and mostly move on to other software systems and join other companies. Additionally, there is no way to see the \u201cbeauty\u201d of proprietary Basecamp without examining the application code, infrastructure and, of course, the people culture at 37 Signals. We have to take their word for it. So while I think a monolith can be magnificent and or even majestic, it is a very tall order to ensure that remains beautiful as time marches on. <\/p>\n

      There is a severe downside to monoliths in commercial companies. They are not normally open sourced and therefore they cannot subject to independent critical review, not even to the structural engineering, security and compliance types. I suspect that the beauty of a monolith is in the eye of the architectural beholder and therefore it can be used as leverage like a big stick to beat the development staff with it. \u201cThis is how it works here, this is what we have and this is how we are going to do it still, in order to get the delivery out of the door for milestone X\u201d.<\/em> In other words, in their opinion, beauty does not pay the bills, the huge salaries and is unworthy of their customer’s attention. So in the end, we eventually finish up with ugly monoliths and they persist until a critical juncture, which might be business model failure and disruptive entries from the competition.<\/p>\n

      Symptoms for an ugly Java EE monolith application <\/h2>\n

      A Java EE monolith is unhealthy when it exhibits the following characteristics:<\/p>\n

        \n
      1. It was not written from the beginning (5 years ago) as a modular application where modules are self-contained with high cohesion and reduced coupling\n<\/li>\n
      2. \nThe original architects have left the building and stage; and the current owners have no clue about deep internals\n<\/li>\n
      3. It is a representation of the political infighting status, managerial power and intrigue\n<\/li>\n
      4. The owning organisation is resistant to change of the monolith. Worldly external consultants and contractors have a tough time getting management to listen<\/li>\n
      5. There is a distinct lack of unit tests, sometimes with less 5% coverage and integration testing is particularly hard, because agile development falls one or two SCRUM sprints beyond the current one<\/li>\n
      6. It is extremely hard to get around hard code dependencies such as customer detail, test databases and limited environments. In other words, testing the simplest code turns into a nightmare.<\/li>\n
      7. The monolith has severe dependencies on legacy libraries from a decade or more years ago: Struts 1.x, WebWork, Tapestry, Spring Framework 1.x or 2.x and exhibits horrendous XML configuration<\/li>\n
      8. The standard build chain is complicated and takes a very long time<\/li>\n
      9. Even if management suddenly gave the green light to the entire development team today, there would be a huge mountain to climb to adopt Java SE 9 \/ JDK 9, because the application code has not evolved much beyond Java SE 6 (2007).<\/li>\n<\/ol>\n

        All of these problems reflect the company\u2019s origin, the culture and people organisation. We will them in an future episode. So there you have it, Ye Olde Monolith<\/em>.<\/p>\n

        [Que horrific laughter from legendary actor Vince Price<\/a> at the end of Michael Jackson’s Thriller<\/a>]<\/p>\n

        +PP+<\/p>\n","protected":false},"excerpt":{"rendered":"

        As a professional software engineer, you probably have encountered an architectural stumbling block known as a MONOLITH#1. A monolith is essentially a single tier application in enterprise architecture. Forget about the traditional break-down of Java EE application, where they break down an Uber-WAR file into presentation, business-logic and database tiers. That is just gloss over […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[28,123,75,59,31],"tags":[],"_links":{"self":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2260"}],"collection":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=2260"}],"version-history":[{"count":26,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2260\/revisions"}],"predecessor-version":[{"id":2302,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2260\/revisions\/2302"}],"wp:attachment":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=2260"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=2260"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=2260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}