DMS Platform

A management tool that gave a publicly traded technology company with over a thousand employees control over the data flowing in from every corner of the organization.

9:41

Control over a growing data warehouse

A publicly traded technology company was centralizing data from all departments and systems into a new data warehouse. The challenge: how do you give employees control over that process without requiring them to write SQL?

The answer: a web application for managing the data warehouse. The tool makes it possible to connect data sources, configure and monitor imports, detect and anonymize personal data, and log everything in comprehensive audit logs.

The scale was considerable: data from APIs, SQL Server databases, PostgreSQL, and various other sources, originating from an organization with over a thousand employees and millions of data points.

Productivity tool for the data warehouse

The import tool pulled data from various types of sources across every corner of the company. Managing this was entirely manual: every new source required development work, monitoring was reactive rather than proactive, and privacy checks were only performed after the fact. The challenge was to streamline this complex and critical system and make it accessible:

1. Diversity of data sources. Each source has its own connection logic: APIs with endpoints and keys, SQL databases with queries, files with specific formats. The interface had to present all of this uniformly.
2. Privacy compliance at scale. With data from every corner of the company, virtually every source unintentionally contains personal data. A built-in PII detection tool had to identify and anonymize this in accordance with GDPR legislation.
3. Monitoring and audit. At this scale, every import must be traceable. Status monitoring, error reporting, and alerts had to be available in real time, with complete audit logs for compliance.
4. Access control. Not every employee is allowed to view or manage every data source. Fine-grained authorization was essential.

The solution

A central management system that makes the data warehouse accessible to the entire team:

• Unified configuration interface. For connecting diverse source types (API, SQL, file).
• Import monitoring dashboard. With real-time status, error handling, and alerts.
• PII detection and anonymization tool. For automated privacy compliance.
• Comprehensive audit logging. For full traceability of all actions.
• Role-based account system. With granular access rights per source and function.

Central interface for data management

The tool became the central interface through which the data team manages the growing data warehouse. Connecting new data sources went from being a technical project to a standardized process.