Design & Development

Moving Data May Become Software’s Best Feature

The need to integrate data among software tools is growing. For ecommerce platforms, streamlining that integration may become essential.

The need to integrate data among software tools is growing. For ecommerce platforms, streamlining that integration may become essential.

Omnichannel commerce, headless CMS, NoSQL databases, edge computing, big data, you name it: The commerce technology stack is changing. And the ability to move information from one system or platform to another is, perhaps, the essential feature mid-sized and enterprise businesses should look for in software providers.

This may be particularly true for brick-and-click retailers and multichannel companies. Often these businesses need to sell products online, from a physical location, and through an outside sales team.

In each of these scenarios — ecommerce, brick-and-mortar, outside sales — both the business and its customers need access to product information, inventory levels, price updates, order history, account information, and more.

This need for shared data is complicated by offline needs. For example, if a shopper is browsing an ecommerce app on a train, she needs to be able to keep shopping, even completing a transaction, if the train goes through a tunnel and her phone loses its data connection.

Monolithic Platforms

For some time, monolithic ecommerce platforms and whole-business software suites have been leading choices for mid-market and enterprise businesses. In part, solutions such as Microsoft Dynamics, SAP, NetSuite, Salesforce, and even Magento are popular because they combine so many software tools.

These sorts of platforms are helpful for many businesses. Specifically, if a company sells only online or just in physical stores, the services work well. They may also work well when a business uses several of the built-in software modules, such as Microsoft Dynamics’ finance, retail, and marketing modules.

However, given new software options and the aforementioned changing technology stacks, some business may require best-of-breed solutions, connecting otherwise unrelated software tools and technologies. This is possible only when information flows easily between these tools.

Integrating large software suites can take months or years. But emerging tools can accelerate the process.

Couchbase

Couchbase, a NoSQL database provider, specializes in edge computing or the idea of storing information close to its users. Its architecture is lightweight and easy to work with.

Take, for example, an omnichannel retailer in the northwest United States that sells in physical stores, online, and through outside sales teams. The company is rebuilding its point-of-sale system using Couchbase.

In stores, it will place a Couchbase server on each register, so the full catalog of more than 130,000 products is available locally. Transactions are also stored locally and passed up to a store-specific Couchbase instance in real-time. That database updates the company’s cloud databases on Amazon Web Services every few minutes.

This arrangement is not new. The chain was doing this with another database, but what is new is the ease at which it is done and the speeds.

Moving the full catalog takes a few milliseconds.

If for some reason a register loses access to the internet, it will continue operating as usual, eventually sharing its transaction information when it reconnects.

If for some reason a register loses access to the internet, it will continue operating as usual…

Couchbase will also be used for the mobile apps that the company’s sales team uses in the field. A salesperson or even an installer (depending on the product) will have access to the company’s full catalog and the ability to process transactions online or off.

MongoDB

But the company isn’t using Couchbase for its product information management. Rather, it had previously built a custom PIM using MongoDB, another NoSQL database. MongoDB works well for product information storage. This is especially true when compared to a relational database such as MySQL.

A large product catalog can be a problem for MySQL. The challenge is that products often have different attributes. A power tool might need an attribute for “battery wattage,” but a t-shirt does not.

In a MySQL database, each row must share every column. So if a business were using just one table to store products, there would need to be a column for “battery wattage” included and left empty for every t-shirt and a “shirt size” column (think small, medium, or large) for every power tool.

To avoid tables with thousands of columns, platforms such as Magento break up the information over many tables, using an entity, attribute, value — EAV — system to keep everything organized. While this does work, it may not perform as well as MongoDB. EAV was helpful before NoSQL databases were available.

MongoDB is a NoSQL, document database. It does not require every product to have access to every column. Rather, it stores only the fields that each product needs.

What’s more, moving information from MongoDB to Couchbase is easy. This development team from the previously-mentioned retailer wrote a simple PHP-based script to update Couchbase every five minutes. Composing this script took just a few minutes.

GraphQL

MongoDB stores information in BSON — binary JSON. For many developers, this format is familiar and easy to use. It makes sharing the information stored in MongoDB easy, especially using GraphQL, an open-source query and manipulation language for application programming interfaces.

GraphQL is an example of a technology that makes moving data between systems relatively easy. This is an example from Shopify.

GraphQL is an example of a technology that makes moving data between systems relatively easy. This is an example from Shopify.

Shopify, which powers something like 800,000 online stores, has a GraphQL API. And sharing information from a MongoDB-based PIM with a Shopify-based website is simple.

For a private app, one that is built for a single Shopify store, authentication is handled with an HTTP header. A business has access to add or update products, retrieve order or customer information, or even update pages or blog posts.

Headless CMS

This same GraphQL API could also be used to connect a headless content management system to Shopify.

Imagine a business that is using ButterCMS or Contently to create its marketing copy and long-form content. Either of these headless CMS solutions could be integrated with Shopify via GraphQL in a day or less.

This applies not just to headless CMS, but also to many kinds of software. Lots of ecommerce platforms and point-of-sale systems have tax modules, varying levels of fraud detection, and even built-in search. But these platforms may not have the best tax solution, fraud prevention, or search for a given business.

…these platforms may not have the best tax solution, fraud prevention, or search for a given business.

This is why large platforms allow extensions and integrations. But even the largest systems or ecommerce platforms may not have an existing integration for every tool a business wants. For example, an ecommerce company might have integrations for:

  • Sales taxes,
  • Fraud detection,
  • Site search,
  • Order management,
  • Customer service,
  • Email,
  • Content.

Moving Data

In each of these examples, the key is the ability to move data. The ease at which order information, customer information, or product information flows between one application or database to another is essential for fast, modern integrations.

For some businesses, connecting otherwise unrelated software, services, and technologies may even be a competitive advantage. And the ability to move data is at its core. Thus, when a business considers a software platform, access to its database or API may be more important than the platform itself.

Armando Roggio
Armando Roggio
Bio   •   RSS Feed


x