Building a Content Analytics Reporting System

Our customer

Trinity Audio is a company that specializes in developing an AI-driven ecosystem of solutions that help manage audio experiences for publishers and content creators. These solutions encompass a wide range of features, including voice editing, content discovery, virtual assistant skills, and data analytics among many others.

The obstacles they faced

The customer wanted to effortlessly generate dynamic real-time reports for their solution by conducting a comprehensive analysis of the large volumes of data about content performance, such as loads and clicks.

How we helped

Romexsoft helped to develop a scalable yet cost-effective reporting system with flexible architecture for data pipelines. This system was designed to process and analyze the required content data of the client’s solution.

Data streaming

• Integration of Apache Kafka was the opening move to handle real-time data effectively. This approach delivers horizontal scaling, ensuring that as data volume grows, Kafka can handle the load without major architectural changes.
• Apache Spark Streaming implementation was employed to consume and process real-time data streaming through Kafka. The inherent ability of Spark to process large data volumes with low latency was instrumental in handling live stream data for this type of solution.

Data storage

• We used Apache Hive infrastructure as a data warehouse for the gathered historical data. It ensures information managing and processing into a readable and structured format for query and analysis.
• The processed and aggregated data were then stored in PostgreSQL as a source for the reports generated by the system.
• Raw data are stored in Amazon S3 object storage service to ensure the cost-effectiveness of the reporting solution.

Data processing

• Trino (PrestoSQL) provides the ability to join historical datasets about content performance (from Hive, PostgreSQL bases, and S3 raw data) with the advertising data from relational databases.
• Amazon QuickSight reports, which showcase required content metrics for ensuring data-driven decision-making from the side of the client.
• Custom dashboards, which get the data from PostgreSQL and Hive databases, represent the usage of the solution and specific content consumed by its users.

Technology stack

• Apache Kafka
• Apache Spark
• Apache Hive
• Trino

Content Analytics Reporting System – Architecture Diagram

• Data-driven decision making
  Our solution presents required data in an intuitive visual format, enabling faster and more insightful analysis, which in turn provides data-driven decision-making.
  

• Streamlined business operations
  Centralized data unification has eliminated data silos and provides a comprehensive, coherent view of business operations, simplifying data management and analysis.
  

• Effective business strategizing
  Our approach accelerates the development of more effective business strategies by achieving comprehensive reports of content performance.
  

• Enhanced insights from the data
  These insights enable a deeper understanding of trends, patterns, and correlations by visualizing real-time and historical data.
  

• Performance and cost optimization
  The implemented solution ensures performance optimization while minimizing the costs of its cloud infrastructure at the same time.