Datastores: An Overview of Common Types

A datastore is not the same thing as a database. In fact, a database is considered a type of datastore. A datastore is a repository for storing collections of data, like a file system or directory. There are several types of datastores, some which are legitimate databases and others which should be used for other purposes. In this first installment of our series, we will give a brief overview of the most common types of datastores.

SQL Database – Structured Query Language is a domain-specific language used in programming. It’s designed for use with data in relational database management systems or stream processing, and is useful in handling structured data. SQL introduced the concept of accessing a multitude of records with a single command, it eliminates specification for how to reach a record and it is user-friendly with no long-term technical training required.

Object Database – Object databases are management systems where information is represented as objects. Those objects are used in object-oriented programming. These databases differ from relational databases which are table-oriented because they are a psuedo-hybrid of both approaches.

Edge Database – Edge databases have features that allow applications to focus on adding business value instead of just integrating, collecting and aggregating data in the edge and sending the result to the cloud. The features are around automatic data integration with devices, automatic data collection and aggregation, and automatic synchronization of data.

Log Stores – Log stores are not databases per se. Log stores literally record events that happen in an operating system or messages between different users of a communication software, called a transaction log. There’s no additional data here, everything is written to a log file which could then be searched for a specific event or message thread.

Caches – A cache is hardware or software that is used to store data in a computing environment on a temporary basis. A cache is not meant to be a long-term use database, but rather an easy access point for frequently accessed data that is stored temporarily. It’s separate from bulk storage, so it moves a bit faster, but only with small amounts of data that is meant to be ephemeral.

Queue Stores – Queue stores are a collection of data maintained in a sequence. The data can be modified by adding to the rear or removed from the head of the queue. This data does not need to remain in a specific order to be worked with and can be pulled or added to in any sequence.

Flat File – A flat file database stores data as a plain text file. Each line of the text holds one record. Fields are separated by delimiters. Flat file systems become more efficient as more data is added.

NoSQL Database – NoSQL references any not SQL databases, which means it can be more than one system. It provides a mechanism for storage and data retrieval modeled in means besides tabular relations.

Time Series Datastores – A time series database is optimized for storing and serving data in chronological order. The software system serves the time series through associated pairs of time and value. This can be referred to as profiles, curves, traces or trends. These timestamps provide critical context for each data point in how it relates to other data points.

Search Appliances – A search appliance is not a datastore, but a type hardware or most often software appliance that is attached to a data store. It’s purpose is to index content, somewhat akin to a how a search engine works. It doesn’t or at least shouldn’t store the data, but is a tool to search information about the data.

Datastores can be complicated and difficult to understand. Some stores are better for one concept or data flow and not another, and it’s important to recognize when to use which type of store. Employing multiple types of datastores can be helpful if you need different ways of sorting and locating data. As this series progresses, we will discuss each type of datastore, what it’s useful for, what it’s good at and when it’s best to use it and most important when to not use it.

About the Author

Pieter VanIperen, Managing Partner of PWV Consultants, leads a boutique group of industry leaders and influencers from the digital tech, security and design industries that acts as trusted technical partners for many Fortune 500 companies, high-visibility startups, universities, defense agencies, and NGOs. He is a 20-year software engineering veteran, who founded or co-founder several companies. He acts as a trusted advisor and mentor to numerous early stage startups, and has held the titles of software and software security executive, consultant and professor. His expert consulting and advisory work spans several industries in finance, media, medical tech, and defense contracting. Has also authored the highly influential precursor HAZL (jADE) programming language.

Contact us

Contact Us About Anything

Need Project Savers, Tech Debt Wranglers, Bleeding Edge Pushers?

Please drop us a note let us know how we can help. If you need help in a crunch make sure to mark your note as Urgent. If we can't help you solve your tech problem, we will help you find someone who can.

1350 Avenue of the Americas, New York City, NY