Build the back end that captures broadcast media and turns it into searchable text, feeding the media results reports the front end projects present. Written in C, not C++, on Ubuntu 22.04. Never run as root; always use a user account.
Free to air antenna television, web streaming television, radio on AM and FM, podcasts, SiriusXM, and the web side of newspapers, magazines, social media and blogs.
Antenna capture uses a Hauppauge WinTV quad tuner board, whose Linux support notes are the starting point. Television stations carry a network affiliation, ABC, CBS, NBC, FOX and so on, which has to be recorded with them.
From television, four streams: closed caption text, speech to text as separate left and right channels with no video, video text recognition, and image recognition. Lip reading to text is wanted as well; both it and image recognition are unsolved and need designing before they are built.
From radio, podcasts and SiriusXM, speech to text on both channels, kept alongside the audio.
From the web, scraped article and post text.
Every output is named for its station, timestamp and kind, for example a closed caption file as KCNC-TV_2023_04_18_14-45-00_FTAAT-CC.vtt and a radio channel as KOA-AM_Radio_2023_04_17-15-25-05-RST-Left.txt. Keep that convention; the front end depends on it.
Import everything into the MySQL relational database that the media results reports are built from, and write a copy to the archive directory, compressed but at high quality. Start with three days of retention, auto-deleting from both MySQL and the archive drive, and make the period configurable by an administrator rather than fixed in code.
Buffer overflows that crash the processing, which is the main reason this work is being proposed again. Cookies need deleting automatically on the scraping side.
FFmpeg for the television and radio processing. For speech to text, VOSK, PaddleSpeech, Otter, Whisper or Athena. For video text recognition, Tesseract. For image recognition, Cloud Vision, Amazon Rekognition, Clarifai, CVEDIA or Roboflow. For web scraping, Apache Nutch, Scrapy, Apify, pyspider or StormCrawler. Microsoft publishes a lip sync sample worth reading for the lip reading work. All of these are suggestions to evaluate, not decisions.
Develop against one source of each kind and let an administrator add the rest later: WISH or CNBC for web streaming television, KCNC for antenna television, KOA for radio, a daily news podcast, Bloomberg Radio on SiriusXM, and WBBM Chicago for YouTube.
This is a back end, so it has nothing to show on its own. Contact the teams working on the front end projects and present through them, generating real media results reports from this data on the deployed site. Archived processed media data covers what is kept long term, and details streaming buttons covers what a user does with a segment once it is there.
Language translation of the processed text is also wanted.
To help set up a real world computer software development work experience and to work with other capstone teams.
To take all of the media Television Radio Webstream Podcast and capture and processes the data and work with the frontend teams to make sure each media data can become demonstratable from the frontend. To present the frontend needs to be up on AWS Amazon on wwwl.digiclipsproject.com/subscriber and on www.digiclipsproject.com/generalpublic
Software to be familiar with would be Ubuntu-Mate 22.04, C, nodejs, react, angular 20 https://angular.dev/ typescript, mysql relational database, putting up on AWS Amazon Lightsail.
We have students work remotely on our local system through digi-frontend for development testing debugging then put up on DigiClips Github and for final deliverable and presentation put up front end on AWS Amazon Lightsail www.digiclipsproject.com
Software to be familiar with would be Ubuntu-Mate 22.04, C, nodejs, react, angular 20 https://angular.dev/ typescript, mysql relational database, putting up on AWS Amazon Lightsail.
We have students work remotely on our local system through digi-frontend for development testing debugging then put up on DigiClips Github and for final deliverable and presentation put up front end on AWS Amazon Lightsail www.digiclipsproject.com
Bob Shapiro
