The Importance of Proper Database Use

There are advantages and disadvantages for every type of datastore. Most developers and coders are generally taught how to code and use databases with SQL as a basis, although more modern-day devs and coders may have learned NoSQL as a basis. When you use an SDK for a product that makes it look and act like something you are familiar with, it’s important to realize that you aren’t actually using that system in its native form. Something is wrapped around it to simplify it into a language in the context you are used to.

Many developers start off thinking that everything should be in one particular database, the one they know and understand. Until they hit a limitation of that database. So, they research and discover something new that solves a problem or set of problems they have, and so they decide that this is the new thing to use for everything. Except whatever this new thing is, wasn’t designed to do that either.

One of the most important things for developers and coders to remember and realize is that datastores and data storage solutions all have very specific names. Their names generally indicate what they are to be used for and what they are good at. Log stores log events. Search appliances are for searching. Time series keep data in chronological order. Geospatial datastores are for data locations. Relational databases are good at relating data to other data. Object stores organize data like a pyramid with the object on top but are not efficient at relating that data to the data of other objects.

Each type of datastore or database is good at something specific. If you misappropriate them, you really make things harder on yourself.

New or novice developers sometimes do not understand that most of the failings and fragility of data stores will not rear their heads when you are building and testing and using small amounts of data. Some stores are meant to work with large amounts of data, but you won’t realize there are problems because the amount of data used during building and testing is small. And sometimes problems don’t rear their heads until you’re trying to do something more complex.

There’s been an abstraction away from databases in coding languages. Enterprises and some of the larger database vendors have proliferated this magic around databases where you needed a special DB admin to do anything. Which meant that developers were steered away. And now architecting out data schemas has become a lost art; which developers don’t need to learn because you can use an SDK to talk to the data store in whatever language the dev is familiar with. The problem is that they don’t have to go down to a deeper level and understand what is happening inside the database or datastore.

Companies and database vendors have adapted to this way of thinking. They obviously want people to use their product, so they build the datastore and then build the SDK to make it function like an SQL or object or NoSQL database. So when the dev is dealing with it, it looks like something they are familiar with. But under the hood it might work completely differently. Because it looks familiar and it solves a problem they were having, they don’t consider all of the drawbacks and other problems the database might cause.

This brings us full circle, now. It’s all about specialization, which is a problem across computing in general. Coders and developers might only learn one or two languages and try to force everything they do into those formats, but there are other tools out there. Similarly, with datastores, there are many different types. Some are good at one specific thing while others are good at something else. You wouldn’t build a house using just a hammer and nails.

The best way to handle data storage is to use multiple types of databases in conjunction with each other. This minimizes fragility because you are using each database in its native form, and it increases your ability to locate problems early and prevent machines from breaking.

About the Author

Pieter VanIperen, Managing Partner of PWV Consultants, leads a boutique group of industry leaders and influencers from the digital tech, security and design industries that acts as trusted technical partners for many Fortune 500 companies, high-visibility startups, universities, defense agencies, and NGOs. He is a 20-year software engineering veteran, who founded or co-founder several companies. He acts as a trusted advisor and mentor to numerous early stage startups, and has held the titles of software and software security executive, consultant and professor. His expert consulting and advisory work spans several industries in finance, media, medical tech, and defense contracting. Has also authored the highly influential precursor HAZL (jADE) programming language.

Contact us

Contact Us About Anything

Need Project Savers, Tech Debt Wranglers, Bleeding Edge Pushers?

Please drop us a note let us know how we can help. If you need help in a crunch make sure to mark your note as Urgent. If we can't help you solve your tech problem, we will help you find someone who can.

1350 Avenue of the Americas, New York City, NY