How our new storage data system is set to turn PRISMA’s problem solving supersonic

It may not have been apparent to the naked eye, but recently PRISMA concluded what we expect to be a major game changer when it comes to how our platform stores and processes data.

Our platform-wide Event Migration – the result of more than a year of hard work spanning several departments – is the first step in the process of Platform Refactoring. And its impact is likely to be significant.

So what is an Event Migration, or indeed an Event, and what does any of this have to do with the storage and processing of data?

We’ll come to that in a moment, but first allow me to present a handy metaphor. When it comes to storing information, imagine for a moment that you can do so in either clay or Lego – in other words, in one unwieldy lump or in self-contained pieces. Of the two, it’s arguably the latter that offers more advantages. Think versatility, security, flexibility, and so on.

And when it comes to PRISMA’s ability to store information, there’s no doubt that we’d much rather be a supersonic Lego spacecraft than a brittle clay tablet.

What’s an Event? 

Let’s start by breaking the subject of Event Migration down to its component parts and explain what we mean by an ‘event’.

In essence, an event is a cohesive piece of data; a self-contained digital fact with a complete process that can be analysed and understood in isolation.

Let’s turn to another metaphor. Imagine you go to the bank and deposit ten euros in your account. Later, you log into your banking app and see the ten euros listed as a transaction. That is an event. Next, you walk to the coffee shop and buy yourself an iced latte (it’s a sweltering summer’s day) for three euros. You check your app again and the transaction is right there. Another event.

Similarly, a bid in a daily auction, an FCFS booking, a trade proposal, or a company registration, are bundles of data that can be organised as events. As such, they must each contain the necessary information that distinguishes them from other events and relate it back to them.

So, a short-term bid event includes information about the user who placed it, the requested amount of capacity, the offered surcharge, the auction in which it was placed, and so forth.

What’s an Event Migration?

An Event Migration – or, more precisely, a Data Migration to Events – is the process of transferring the legacy data of a system into the digital events described above. In other words, it’s the creation and structuration of events based on already-existing data.

Information need not be stored in events. You can record data in large, monolithic databases in which processes are not separated into discrete, yet interconnected, cells. In that case, though the data is still usable, it’s harder to parse and manipulate information. Monoliths, as the name implies, are rigid: structured, but disordered.

So, in the case of our recent Event Migration, we took all the data from our monolith and split it into smaller events, which are at once easier to work with and harder to break. It’s as if we carefully broke ancient clay tablets with years of engraved data into sensible Lego pieces and rebuilt the entire platform piece by piece.

The Challenges

This project was nothing short of an archeological excavation. After all, we had to descend into the catacombs of the platform to retrieve the clay tablets. We then needed the aid of a platform expert who could interpret the ancient script on their delicate surfaces. Finally, we required a large team of engineers to build the digital machinery that could extract the data from the tablets and convert that information into sensible bricks.

It was crucial, therefore, to map all of the processes and transactions in the PRISMA database and determine what events could be created, what data each should contain, and how they connect to one another. A true PRISMA stalwart, Marcus Wadewitz, led this interpretive task, while our talented team of engineers wrote the code that would extract the data and create the events.

After several months of preparation, the Event Migration itself happened on Tuesday, August 2^nd. It was a silent, but gargantuan, effort. A multi-departmental taskforce was formed; our Product Owners oversaw the project; our Customer Success Team dutifully monitored the platform to ensure nothing went wrong during the migration; and engineers conducted the migration and remained ready to respond to any errors.

We were all a little tense throughout the day. While the process had been meticulously tested, some factors always remain unpredictable. In the end, however, the platform was down for no more than 30 minutes and only a few functionalities were temporarily turned off – and even then, for a shorter period than expected.

Needless to say, we believe the Event Migration was a resounding success.

The Benefits

Now, you might well ask yourself, why go to all this trouble? The answer is simple: at PRISMA, we always strive to help our users achieve their goals. For us, it’s imperative that we uphold our commitment to offering innovative, iterative, state-of-the-art solutions to the current challenges facing the gas market. To do so, it’s essential that we modernise our platform, making it more dynamic, flexible and intuitive to create new functionalities, address users’ needs, and adapt to unforeseen market situations.

That’s precisely what our new Event Storage Structure will enable us to do in the context of our ongoing Platform Refactoring. It will allow us to store your data in a safer, more intuitive manner; fix problems with greater speed and accuracy; craft new functionalities more easily; and respond to market changes and pressures faster.

In short, PRISMA is now more versatile than ever before. And versatility is the only answer to volatility.