Data Pipelines to the rescue
This is a continuation of part 1 of my post. As I had written, a healthcare CRM that isn’t constantly updated with data from these multiple source is like a car without fuel: you can push a lot of buttons, but you won’t be going anywhere. To crack this problem, a healthcare CRM needs to ingest, clean, and unify data from these multiple sources. All of this needs to be done in a repeatable and reliable way to produce a unified view of these disparate data sources. This is a job for data pipelines:
Data Pipeline: (noun) a repeatable sequence of actions that take data from an initial state then process it and transform it into a final usable state
At SymphonyRM, we made a strategic technology decision to implement our data pipelines on Airflow, an open-source data pipeline framework released by AirBnB. It lets us easily build, execute, and monitor complex workflows that form our many data pipelines that we run for clients.
Even choosing a great data pipeline framework like Airflow isn’t enough though. On top of Airflow we’ve made a huge investment in reusable components that solve common challenges in unifying healthcare data. As just a few examples, we have modules to flexibly import data files into databases, perform score-based matching of patient data where unique IDs aren’t available, normalize care gaps from multiple payers, ingest schedule data into standardized form, run algorithms that prioritize next best actions, etc.
With this infrastructure we’re able to very easily take in data from many sources, present clients with a unified view, and run scoring algorithms to guide clients to the next best actions throughout their business. Since the integration is done with simple flat files, clients find it very easy to get up and running. (We like to say that we have a core competence in processing CSV files. As low-tech as CSV files are, they’re widely available and easy to create from almost any system.) Once clients have this unified view of data and guidance on next best actions, they really start to transform their business and move important metrics like care gap closure, leakage, and provider utilization.
Getting ready for Machine Learning and Artificial Intelligence
There is a lot of promise in applying machine learning and artificial intelligence to healthcare data sets. Yet those technologies are very sensitive to having clean data to train models and neural networks. Work on data pipelines will help fuel your investments in ML and AI technologies, too.
Languish or lift-off?
It turns out that the most important part of a healthcare CRM system isn’t the CRM portion. Without sophisticated data pipelines to unify data from many sources you won’t be going very far with your CRM implementation. However, if you make some smart technology choices and put the right data pipelines in place, you can fuel your CRM with actionable data and get ready to blast off. Test the SymphonyRM HealthOS platform to turn data into action.
Joe Schmid is SymphonyRM's Chief Technology Officer. His experience includes Platform Lead at Victrio/Verint, Director Solution Engineering at Voxify/24-7, and Platform Engineer at BeVocal/Nuance. LinkedIn: https://www.linkedin.com/in/joeschmid/
