Databases have been used to generate web pages since the early web. Databases are well suited to managing data. When the web became dynamic, databases continued to be use for data persistence. By 1993 the web as an application platform was born1 and with it a good platform for providing useful organizational and institutional software.
By the mid 1990s the Open Source databases like MySQL and Postgres were popular choices for building web applications. It is important to note neither MySQL or Postgres spoke HTTP2. To solve this problem many people wrote software in languages like Perl, PHP and Python that ran inside the popular Apache web server. It was a pain to setup but once setup relatively easy to build things that relied on databases. This led the web to explode with bespoke systems. By the late 1990s and early 2000s the practice of “mashing up” sites (i.e. content reuse) was the rage. Bespoke systems took advantage of content reuse too. Yahoo Pipes was a very interesting expression of the “mashup culture”3. Yahoo Pipes inspired Newt’s data pipelines. Eventual the bespoke systems gave way to common use cases4. A good example of a common use case is Apache’s Solr search engine. Another example is how bespoke content management systems gave way to Plone, Drupal and WordPress.
family_name_1)The evolution seen in this abbreviated timeline illustrates a growth complexity, implementation and expectations around web development. In 2024 many organizations and people appear to operating with a maxim, “the web is complex, my tools need to be complex, I create complex things”. This maxim is not sustainable in software.
The seeds of simplification are out there. They’ve cropped up at almost every turning point in the web evolution. Sometimes the seeds of simplification can actually result in more complexity, e.g. my experience with early NodeJS work was liberating. I went from projects where I worked in five or six programming and configuration languages to four language (e.g. JavaScript, HTML, CSS and SQL). That same liberation also laid the foundation for NodeJS+NPM which ushered in the assumption of a complex ecosystem. Static website deployments came back into vogue (in part due to cost advantages of using S3 buckets) but some of the static site generators, e.g. Jekyll, seemed to have missed the boat on simplifying things. These types of simplifications came from developers for developers in many cases. One simplification freeing conceptual bandwidth to facilate an increase in complexity.
On the other hand simplifications which had multiple motivators seem to stick around a while. Static sites are common practice for libraries since they provide a robust platform for distributing information and are also low cost to support (rental of an S3 bucket is cheap for small files).
You also see other forces encouraging a rethink. The race to the “cloud” has lead to a landlord’s market place. For commercial software it is difficult to “buy” software but often forcing us to rent7.
I think it is high time to focus our simplification at all levels. Getting to simple in part is an engineering problem, in part a human organizational problem and significantly a “market force” problem. As humans we can take steps to change that if we choose to.
Web applications proceeded to eat all the venerable green screen systems they could find. Today’s web is mired in surveillance tech and complex solutions. It has drifted far from Sir. Tim’s vision of sharing science documents. We need to refocus on the “good ideas” and jettison the complexity that came with the surveillance economy. Newt can be part of that solution. Develop your Newt application with consideration for others.↩︎
HTTP being the protocol the communicates with. Essentially at the time RDBMS spoke a dialect of SQL as the unifying language. The web of the time understood HTML and to a certain degree XML. By 2000 people were looking for something simpler than XML to move structured data about. JSON quickly became the answer.↩︎
The basic concept was to make it easy to work with “data feeds” and combined them into a useful human friendly web pages. It even included a visual programming language to make it friendly to the non-programmer crowd.↩︎
If a use case is solved reliably enough it becomes “off the shelf” software.↩︎
E.g. Prototype 2005, Mootools, YUI and jQuery 2006.↩︎
Ryan Dahl releases introduces NodeJS. This quickly replaces Rhino, Narwhal and Jaxer JavaScript server side efforts↩︎
Example try buying Windows, macOS, or Adobe Photoshop, we rent the software only.↩︎