Finishing in half the time

Every time I start out a new project, I have to spend weeks simply getting the skeleton for my project up running. This is a highly repetetive process, and although I’m getting fairly good at it, the fact that I’ll need to do this repetetive task over and over again, is actually quite annoying to be honest. Simply wiring up Fluent nHibernate, Ninject, Mapster, and all the libraries I want in my project, often requires days of searching for solutions on Google and StackOverflow.com. This process also happens to be of such a nature that I rarely seem to remember exactly how to do it correctly from project to project, and the statistical probability of that I’ll do something wrong, is unfortunately very high.

For instance, in my particular industry, which happens to be ForEx and finance, there’s this concept of KYC, which means Know Your Client. For me as a developer, this implies I’ll have to create some controller endpoint, that can accept uploading of files, such that my employer’s clients can upload images of their identity cards, and/or passports. The arguments for this, is to avoid being used to do money laundering on behalf of dubious clients, whom for some reasons are doing dubious things. Regardless of the legal arguments, the task for me is the exact same; Create an HTTP REST controller endpoint, where I can accept files, store these files associated with some client, and make sure the files are secured, such that others cannot access them in any ways. Every time I create a system for my employer, the task is the same. Simply making sure this is correctly created, such that it’s secure, and doesn’t in any ways compromise the safety of our clients, often require days and weeks of coding, every single time I start out a new project.

The problem is that this type of solution doesn’t easily lend itself to libraries either. As far as I know, there doesn’t really exist good libraries allowing me to avoid having to do this task. First of all because every single time I implement this feature, it requires slightly modified code for that particular application. Some apps might want the clients to see the files they have uploaded, and possibly even upload new versions – While other apps only want the client to be able to upload a maximum of 4 files, etc. Other apps might require the uploader to be authorised, others don’t – Etc …

There is no way that I know about to intelligently create libraries that are flexible enough to permanently solve this problem. Still the code is similar enough every time to imply that it’s a problem that should be solved once, for then to never be looked at again. There are also tons of similar problems I face, that are common problems, but doesn’t lend themselves easily to libraries or NuGet packages either, because they require too much customisation to be intelligently reused as libraries. Authorization and authentication comes to mind …

Meet Magic

The purpose of Magic is to provide me with a skeleton for such projects. Basically, a starter kit, allowing me to start out with 50% of the problem already solved, for then to give me extension points where I can modify its source code easily, to accommodate for the particular problem I am currently facing, in whatever application I am currently building at the moment. Uploading of files securely to my web server? Sure, no problem. Already fixed.

With Magic I am able to start out my new projects in such a way that I no longer need to spend weeks applying structure, and implementing these repetetive tasks, I know I’ll have to do every time I start out a new project. At the same time, it’s not a NuGet package, and probably not possible to distribute as one either – But rather exclusively distributed as pure source code. This allows me to modify any aspects of it, by simply editing its existing code. In a way it becomes to my own day job the equivalent of what a box of Tandoori sauce is to my girlfriend. It allows her to rapidly cook Indian food, without having to spend hours creating the sauce. A box of Tandoori sauce has little value by itself, but mixed with some rice, meat, and veggies, it becomes a perfectly valid Indian dish. At the same time, my girlfriend can choose if she wants to add beef to it, chicken, carrots or onions. It doesn’t destroy her ability to think creatively in any ways, it just simply does what she doesn’t want to do, or don’t know how to do. It allows my girlfriend to make an Indian dish in half the time, while it at the same time increasing her ratio of success.

I’ll obviously use Magic myself in my own job. But in addition, I have also created it as a publicly available and open source starter kit for others to use as they see fit. This approach allows me to get valuable feedback about the project(s), and have other developers scrutinise my code, and come with suggestions, which is always a good thing. For open source applications, it’s free of charge to use. For closed source applications I charge €50 for each module in it. This allows you to start out with half the job done, for a handful of EUROs, every time you need to solve similar problems as I need to solve in my job. Arguably doing half your job, making you that more productive, for some 50-150 EUROs every time you start out a new project. Effectively becoming the “Tandoori Sauce of Software Development.”

Check out its source code at GitHub if you wish

Problems currently solved

So far I have created 4 projects. The main project is simply called Magic. The core of Magic provides you with an extendible web API, allowing you to drop in any controller endpoints you wish. It wires up Ninject, nHibernate, allows you to choose your database provider transparently, and does most of the wiring up. Out of the box, it is a simple TODO web API, but the TODO controller is simply there to serve as an example.

Among one of its really nice traits I think, is its ability to rapidly create CRUD controller endpoints, for any type where you need (C)reate, (R)ead, (U)pdate, and (D)elete capabilities. In fact, this process is so simple, you can implement CRUD for any types you wish, literally without coding.

Magic Auth

Magic Auth solves the problem of authenticating and authorising users. It uses BCrypt to hash your users’ passwords, and applies JWT token authorisation to your web APIs with a couple of lines of code. This allows you to simply drop in Magic Auth, for then to start applying [Authorize] attributes to your controller endpoints. It also of course allows your users to change their passwords, and such – But contain no additional “baggage” besides the bare minimums most web APIs require. This allows you to modify the existing User domain model, to apply any amount of customisations you wish. Email field, address fields and phone no fields for instance …?

Magic IO

Magic IO is the project that implements the use case we started out with; Handling files and folders on your server. It allows you to create folders, upload files, download files, and query and delete folders and files on your server as you see fit. It’s effectively a minimalistic implementation of the System.IO namespace from .Net Core. Not entirely finished at the point in time where I am writing this article, but I’ll probably wrap it up during a week or two, and create a release for it.

Magic Email

Magic Email gives you a webmail backend API, allowing you to implement your own version of Gmail if you wish. It features the ability to create POP3 accounts, and then have the system automatically and periodically poll your POP3 server(s) to see if any new emails have arrived. In addition it allows you to easily send rich emails, using MimeKit and MailKit instead of the builtin SmtpServer from .Net Core, which I am sorry to say, never seem to be powerful enough in functionality for my own personal use cases.

Interested in seeing how I can help your own software development efforts? Feel free to use the contact form below.

Advertisements

A Software Factory Process Proposition

There is nothing magical about the creation of software that doesn’t allow us to automate it. In fact, arguably the sole purpose of software, is automation of processes – So why haven’t we been able to automate our own processes? Why haven’t we automated the creation of software itself?

When Henry Ford started producing cars, he would look for places where he could standardise the construction process. This obsession went so far that he jokingly said “You can get a Ford car in any colour you wish; Black or black.” The idea of course, was that the colour black would dry in half the time other colours needed to dry, and that by standardising his colours, he could simplify the process of construction, resulting in that he could produce more cars in the same amount of time, resulting in less expensive cars for his consumers, and higher market shares for his company. As a consequence, an entire colour industry spun of in his trail, creating hundreds of additional companies, who’s sole purpose was to repaint cars to whatever colour the customer actually wanted after they had purchased a Ford. These companies for obvious reasons had an incentive to make sure Ford would gain even higher market shares, and hence such ended up advocated the advantages of purchasing a Ford. By underdelivering, he gained even more milage on his competitors.

If you look at the software creation process, most of our work is repetetive in nature. This is a sign of that our workload in these regards lends itself to automation. Everything that is repetetive in nature, easily lends itself to automation. Hence, arguably 80% of our processes can be automated. Few employees today are more expensive than software developers. If we can automate 80% of a software developer’s workload, we can cut the cost of creating software down by 80%. Hence, by automating our own jobs, an application that would normally cost €100.000, would end up costing €20.000 instead, and we have facilitated for a “software factory” – An assembly line production facility, that allows us to create software faster, less expensive, to more people, than our competitors can do. Needless to say, but such a facility would arguably within months of starting up, literally acquire monopoly on the process of creating software.

What colours do you want for your web API? You can choose; Black or black

Check out Super DRY Magic here

Creating Gmail in 3 minutes using C#, ASP.NET Core and Magic

For those not following me on GitHub, I can inform you that I have created a new “Starter Kit” framework called Magic. Its purpose is to be a kickstarter for your own ASP.NET Core web API projects, allowing you to rapidly get started with creating your own HTTP REST web APIs. In the video below I am demonstrating how I use this to create a webmail backend, for both sending emails over SMTP, and retrieving emails over POP3. The idea is to create commoditised solutions for most common problems, to avoid repeating myself, and hopefully also to encourage you to also keep your code DRY.

The project(s) are Open Source, but a commercial license path exists for those wanting to create closed source apps, which will cost you in total €100 for both projects; Magic Core and Magic Email. In the above video I am using;

Teach yourself DDD in 15 minutes

DDD or Domain Driven Design is one of the most difficult thing to learn. Entire libraries have been written about the subject, and many senior developers have spent decades trying to learn it, often unsuccessfully may I add. Its elusiveness for software architectures, sometimes feels almost like the equivalent of searching for the grail. At the same time, it’s considered the most important concept to know for a software architecture to be considered a senior architect among his peers. However, if you simplify DDD, and create generalisations around its ideas, it’s actually pretty simple to understand.

I have created a Micro Framework for ASP.NET Core web APIs that allows you to browse its 400 lines of code in 20 minutes, and afterwards actually have not only a pretty strong grip upon DDD, but also your own tool to start using DDD in your own solutions. In essence, it’s a starter kit for ASP.NET Core web API development, that creates a template for you, that makes sure you’re following all best practices if you follow a simple recipe as you create your own web APIs.

The basic ideas are DRY, DRY and DRY – As in “Don’t Repeat Yourself”. By following this axiom as deep into the Rabbit Hole as possible, you end up with a framework, that allows you to create your C# apps, literally without coding almost at all.

Check it out for yourself here