Distributed Data Processing System

Home 5G/4G Cellular Network Analytics System Distributed Data Processing System

Distributed Data Processing System

Components

central_server1
Features
  • A Pub-Sub [Publish-Subscriber] Architecture
  • Extremely robust & elastic design
  • Processes 2TB of unstructured audio/video data, every day, from 150 teams [each team having 30 handsets], spread over 100 cities across USA [3 different time-zones], within 3 Hrs
  • Generates executive level KPIs [Key Performance Indicators] & Summary reports. All the reports are available at 12 AM midnight, in a given timezone
  • End to end automated operations with the help of Micro services & Distributed task management system [using Celery & AMQP based message broker RabbitMQ]
Extremely fast processing

2TB of unstructured audio/video data is processed in just 3Hrs

Executive level KPIs

Computation of executive level KPIs

End to end automation

End to end automated operations including remote device management, remote test management & remote user management

UE Sync Micro Service

Features
  • Establishes synchronization between caller UE [User Equipment] and callee UE [User Equipment] devices
  • UEs communicate with this service via RESTful APIs, over HTTP/HTTPS

File Upload Micro Service

Features
  • UEs upload the collected data [recorded audio/video files, layer-3 message files, parsed data etc...] to this service via RESTful APIs, over HTTP/HTTPS
  • This service store uploaded files on to network shared folder
  • After the file upload, it publishes data processing request on to AMQP based distributed message broker RabbitMQ

Notification Micro Service

Features
  • Notifies distributed Celery workers to start the task

Celery Worker

Features
  • Subscribes tasks from Distributed Tasks Queue
  • Accesses data from network shared folder and parses video RTP data
  • Computes video analytics & KPIs
  • Splits audio file & computes audio MOS [Mean Opinion Score] by comparing each audio clip with the reference audio clip
  • Splits video file & computes video MOS [Mean Opinion Score] by comparing each video clip with the reference video clip
  • Save processed data & parsed layer-3 messages on to centralized database
KPIs
  • Audio Quality Analytics [Audio MOS]
  • Video Quality Analytics [Video MOS]
  • Number of video RTP packets sent/received
  • Number of video RTP packets played
  • Number of video RTP packets dropped
  • Number of video frame freezes
  • Number of video frames skipped
  • Amount of jerkiness in the video clips
  • Lip-sync Factor between the video and audio during the call

Web Portal

Test Management

Start/Stop/Monitor tests remotely

Device Management

Add/Remove/Start/Stop/Reboot device remotely

User Management

Add/Remove users remotely

Team Management

Setup/Update/Remove teams

View Reports

View summary reports

Download Reports

Download summary reports

error: