EBU – MDN WORKSHOP 2019 – Day 2

EBU – MDN WORKSHOP 2019 – Day 2

Day 2 of the MDN workshop welcomed us with a sunny weather, that reminded much more of Barcelona rather than Central Europe. From the academical perspective, we enjoyed very nice presentations, among them we’d like to highlight the following.

The first one, entitled “AI tools for audio verification and production, and program analysis”, was done by people from Fraunhofer, company responsible for the MP3 audio format (all hail Fraunhofer!). This presentation aimed at showing how easy can we be fooled by audio alterations and which tools are they developing in order to help reporters in being able to tell whether and audio material is to be trusted or not.

Another nice feature was shown by France Télévisions. They presented some AI improvements that they are running on their broadcast.  The one the struck the most was how are they integrating live automatic sign language translation in animated shows, so that deaf children are able to follow them, given that lip reading isn’t possible with cartoons. In order to do this, they have used some nice AI features such as facial detection to identify the character who’s speaking and speech to text recognition, through NLP techniques, and finally, transform this text into the sign language that will be shown on screen.

The presentation “Experiences from the development of the TV2 Sumo recommendation engine” was a great opportunity for us to see the state of art in recommender systems compare to our actual development of recommendations. Sumo is the streaming platform for Norwegian broadcast corporation TV2. In the last year they have developed a recommender system that is now providing personalized material to the Sumo users with a nice success. Nowadays, there is huge amount of tv shows, movies, documentaries, … ready for people to watch, and more of it is being constantly produced. In this context, it is completely necessary to provide some guidance so that consumers don’t get lost in this ocean of video material. Recommending systems are a great tool to identify material that can be relevant for each individual based on their history and the history of other individuals. At Konodrac, we have developed a recommender system that not only takes into account what have individuals watched in the past, but it also weights in greater importance the latest material that this individual has watched. Recommendations are an extremely challenging task, not only from a technical perspective, but also from a societal one. Questions like “can we beat people’s friends and family recommendations?” or “should we recommend material that only matches the filter bubble of each individual or should we present him/her sources that will expand his/her comfort zone?” are a great challenge for these technologies and will have be faced in the near future. Yesterday’s presentation showed us that our system is very well aligned with other systems and the directions we are taking in order to improve the quality of our recommendations will provide an extremely useful tool for both society and our clients.

Tomorrow’s the last day of this workshop and so far, it has been a great opportunity to listen to very interesting talks and people that are making us raise points and questions that will end up in better features offered in Konodrac!

EBU – MDN WORKSHOP 2019 – Day 1


This week, Konodrac is in Geneva at an EBU event (European Broadcast Union). The event is organized by the EBU Strategic Program on Media Information Management and AI (MIM-AI) and the Metadata Development Network (MDN), a community for developers in which knowledge is shared, current work is presented, feedback is received, and there are opportunities for collaboration in metadata related projects.

Among other things, yesterday they talked about a job done by the Swiss television, RTS, which allows their workers, especially journalists, the search and access of images and videos of people of interest. This is new way of working that moves from traditional search, either extensive or based on written annotations, to one in which you search for an individual, and you get all video frames in the catalogue where that person appears. This system also allows you to place an image of a person and the system, it will automatically return all the videos where person. In a similar way, this system supports images of objects or relevant landmarks. To achieve this, they built a system that uses precalculated neural networks (MRCNN, RESNET-50 or FACENET) that compares the image in question with the whole archive, presenting the most relevant and similar results.

Concerning Konodrac’s lines of work, we enjoyed a presentation that aligned very well with some of the interests that we have right now. Konodrac has been offering television channels the chance to have reliable audience metrics in real time. Recently, we have focused on offering a wide range of categorical variables associated with TV consumption. For example, we can see, from total consumption, what is the fraction of viewers who consume content from the web or from a certain category in a given taxonomy. Having this information allows broadcasts, to better define their audience by answering questions such as “who’s watching?”, “How are they watching?”, “How long are they watching?” … Yesterday’s talk seeks to deepen this analysis, by enriching consumer data with metadata, not associated with consumption but with the broadcast itself. Consider, for example, seeing the name of the talk-show guest who is intervening at a specific time, associated with the audience curve, or the protagonists of the news that is being broadcast. As we are now doing by integrating social networks, automatically incorporating quality annotations from the videos that broadcasts are playing to the tools offered by Konodrac, would allow our clients to be able, not only to answer descriptive questions, but to be able to make inferences on possible causes based on the changes in behavior of their audience.

We will see what our European colleagues offer us that help us to keep learning and to continue looking for ways to improve our product. More on this tomorrow!