Data unification in local government: can creative thinking help bypass complexity?

It’s becoming increasingly complex for local government organizations to keep up with the digital profiles and identities of the citizens, businesses and interest groups they serve. Increasingly comfortable with digital channels, these stakeholders move effortlessly across devices and between websites, social media, chatbots and call centres — but their data doesn’t always follow them.

Data unification across these platforms, and more, would enable public sector service providers to deliver more personalized experiences and relevant services — potentially at a lower cost through more efficient resourcing. Data unification has been defined by some as “the process of ingesting data from various operational systems and combining them into a single source by performing transformations, schema integrations, deduplications, and general cleaning of all the records.”

And if that sounds difficult, that’s because it is. Extremely difficult, in fact. 

Three common challenges with data unification in local government

The main challenges with data unification encountered by local government organizations are rooted in data quality, technology/infrastructure, and culture. 

Before data can be brought together it must be cleaned and deduplicated, multiple types of data must be classified in a way that provides some sort of universal compatibility for information from, for example, the electoral roll, benefits and tax entitlements and communications preferences. Compliance is also a key quality factor, with regulations or legislation such as the Data Protection Act or GDPR tightly governing the ways in which local government organizations can handle personal data. 

Technology needs unifying, too, through a data architecture capable of managing varying data sources, and supported by a DataOps function to manage and continuously improve its performance. Legacy infrastructures can make this tricky and traditional, labour-intensive data extraction methods like Extract Transform Load (ETL) are not sustainable when large data volumes from multiple sources are in play.

Finally, unifying data — and keeping it that way as new technologies such as IoT and new sources of data emerge — demands an organization-wide data strategy. That’s likely to stir up entrenched feelings of ownership over processes and their associated data.

Overcome these challenges with realistic data unification goals

Given the long-term nature of these challenges, local government organizations need to set realistic goals for their data unification projects. Smaller projects, ones aggregating a handful of data sources for departments that have a clear and compelling use case, can yield quicker wins. An example might be providing a dashboard showing all of an individual’s or business’s existing service requirements to frontline staff. These can be useful for building up the political capital needed to aim higher in the future.

They should also seek solutions that can cost-effectively augment the capabilities of their current infrastructure and allow them to bypass lengthy ETL processes (often a huge barrier to entry for data unification). The data virtualization and virtual schema capabilities of modern, high-performance Loading...analytics databases achieve this by working as an abstraction layer across multiple applications, data sources, formats, locations, and even across hybrid cloud environments.

The benefits of unified data in local government

The use cases that unified data unlocks are suitably rewarding for such a high-effort initiative. For example, connecting and analysing data from various citizen-facing platforms enables advanced sentiment analysis. That would allow organisations to get closer to citizens by anticipating their needs. Unified data also provides a foundation for modern applications that can help drive service excellence up and costs down by, for example, allowing citizens to access services on a self-serve basis (reducing resource demands in the process). The Local Government Association cites the power of predictive analytics — for which unified data is essential — to drive preventative approaches to homelessness, debt management, and troubled families.

The organizations most likely to realise these benefits are those with the vision and discipline to pick the low-hanging fruit before going after the roots and branches. From an analytics infrastructure point of view that means prioritizing low-cost/high-return solutions that can accelerate and optimise performance of existing investments.