A Fortune 500 commercial real estate services firm envisioned a data warehouse and analytics platform upon which they could build robust internal software tools to measure company performance at all levels. Designing and mapping the data was challenging enough, but the data needed to track the metrics identified by the C-Suite were owned and maintained by different business units in the organization. This data was housed in many different systems built with many different technologies. Finally, much of the data was sensitive and confidential to the organization, so IT policy excluded the active push of data to external applications.
Lofty Labs built a distributed data pipeline using open source technologies. By avoiding off-the-shelf ETL products, we were able to create a framework designed to grow with the enterprise. Engineers within the firm’s software teams have the tools to integrate new data sources as they come online in a custom way, rather than a one-size-fits all solution that falls short of expected benefits. By applying clever DevOps automations and architecture Lofty engineers were able to satisfy the data synchronization needs of the cloud based analytics tool, while respecting the strict IT policy of the organization. This included isolated private cloud networks and collapsing VPN tunnels that originated from within the enterprises own data centers. The pipeline hosts sophisticated optimizations including active monitoring and alerts, performance dashboards, failure detection, and synchronization redundancy all the way down to the individual data point.
The organization now has a pipeline capable of moving and manipulating tens of millions of records per day into their analytics platform. The analytics platform itself is a API driven interface that empowers the firm’s software teams to rapidly construct analytics applications that take the whole organizations data into account. The platform currently supports a market analytics tool, a deal flow monitoring tool, and a brokerage management tool deployed across the entire organization. Ultimately, the analytics platform grants the C-Suite access to performance statistics that were previously only accessible after a yearly audit. These insights are now available within 24 hours.
To analyze data, it must first be collected and mapped. Lofty Labs specializes in data driven software products and applications. API based architectures allow for systems to communicate without costly direct integration steps. Lofty Labs designs software to leverage well designed APIs. Data that is accessible to people isn’t necessarily accessible to software and systems. Lofty Labs understands how to convert data assets into automated operations.
