HAND Labs
A Deeper Dive into our "Demo or Die" projects - 2023 to Present
Since 2023, HAND (Human & Digital) has been pushing the boundaries of notable real & virtual talent identification, applying HAND’s interoperable, ISO-level identifier in the service of provenance and automation.
HAND Labs is a running record of our documented demos of HAND’s Talent ID in action – in media supply-chain use cases / PoC’s (proof-of-concept) trials.
Case Study: IBC Accelerators
Putting HAND’s Talent Identifier Framework to Work
Since 2019, the IBC (International Broadcasting Convention) Accelerator Programs have aimed to drive innovation in the media and entertainment industry by providing a framework for agile, collaborative, and fast-track innovation. By focusing on hands-on experimentation in a non-competitive, supporting environment, they have allowed participants (both large and small) to learn about new technologies and explore their potential applications and business value in real-world use cases.
Overall, the IBC Accelerator program serves as a catalyst for industry-wide progress, helping to shape the future of media and entertainment.
HAND has, to date, participated in four (4) IBC Accelerators – two in 2023 and two in 2024.
2024
Accelerator Project:
Digital Replicas and Talent ID: Provenance, Verification and New Automated Workflows
Champions & Participants
Champions: Paramount Global, ETC (USC’s Entertainment Technology Center), SMPTE, ITV, Royal Central School of Speech and Drama
Participants: HAND (Human and Digital), EIDR, Videntifier, Wild Capture, Wave Seven, The Scan Truck, EZDRM
Official Project Showcase Page:
https://show.ibc.org/accelerator-project-digital-replicas-talent-id
Deep-Dive Project Microsite: Digital Replica Provenance Info
The Challenge
Proposed by HAND in November 2023 and awarded as one of the winning pitches at IBC Kickstart Day in London on March 6th, 2024 – this timely challenge addressed the evolving disruption within the Media & Entertainment industry in the authentication and management of talent in today’s digital landscape. Specifically, it addressed issues tied to AI-generated content, virtual world representation, and talent provenance.
By offering a distinct and standardized identification framework, this project aimed to streamline the authentication process, ensuring reliable verification of real individuals, virtual avatars, and fictional entities. This POC exploration responded to the urgent need from the media and entertainment industry for an agile solution to automate provenance, thus verifying the authenticity of digital personas and protecting against unauthorized virtual representations.
The POC also aimed to showcase how to minimize legal complexities and enhance transparency in talent utilization through streamlined collaboration and licensing across industry stakeholders, simplifying licensing agreements based on verified identities. In addition, it aimed to show how producers and distributors can easily identify and track the usage of talent across different platforms and multi-media formats, such as streaming services, podcasts, video games, and ensure that the rights and permissions of the talent are respected and enforced to prevent unauthorized and unethical uses of their digital likeness.
The Objective
Creation of a consent-based Digital Replica of a notable legal and natural public figure
Document and measure the security and applicability of the licensed digital Replica Talent ID in a business use case (eg: deal memos, contracts, licensing, verification, tracking of rights, residuals, performance analytics, etc)
Collaborate with C2PA to contribute to standards partners to embed cryptographic metadata in a watermark/fingerprint- as a C2PA Content Credential for the Digital Replica Identity, in a multi-party workflow
Explore Key Provenance information about the digital replica’s chain of custody with Champions responsible for NILV (name, image, likeness, voice)- eg: Talent Management, Studios, Broadcasters
Test new workflows that include tagging and labelling with a resolvable, unique and interoperable public ID
The Results
- Created a consent-based digital replica of key Hollywood talent: Evan Shafran (known actor, rapper, DJ, Writer, Comedian, Activist, SAG Member) and a video of the replica
- Registered both Evan Shafran and his digital replica with HAND Talent IDs (which is a resolvable, unique, and interoperable public identifier)
- Generated C2PA Content Credentials (CR) hash for the video of Evan’s replica with both Content (EIDR ID) and the Talent ID (HAND) as cryptographic source metadata using EZDRM technology
- Created an “Asset and Metadata” workflow – tracking a digital replica from Project Greenlight, to Capture, Creation, and Distribution Outputs (multiple potential use-cases in theatrical, television, gaming, advertising, etc. examples)
- Mapped connectors in the proposed workflow to key stakeholders in Accounting & Finance teams
- Demonstrated detection of C2PA Content Credentials metadata in the digital replica media object at distribution end-points
- Demonstrated deepfake detection technology utilizing Videntifier’s forensic media file analysis tools.
Accelerator Project:
Design Your Weapons in the Fight Against Disinformation
Champions & Participants
Champions: BBC, EBU, TV2, Paramount Global, ITN, Transmixr, Media Cluster Norway, YLE, Channel 4, Associated Press, ETC (USC’s Entertainment Technology Center), CBC, IET, Globo, 2G Digital, CBS News
Participants: HAND (Human and Digital), Eluvio, Open Origins, Nagra, Videntifier
Major Media Coverage: Hollywood Reporter: “How the World’s Top News Orgs Hope to Tackle Misinformation as a Historic U.S. Election Looms”
SEPTEMBER 30, 2024
Official Project Showcase Page: https://show.ibc.org/accelerator-project-design-weapons-fight-against-disinformation
The Challenge
The project team set out to develop a comprehensive industry-wide understanding of the challenges faced today by all media outlets in identifying disinformation, helping audiences to identify trustworthy news, Info and images as well as the lengths that news organisations go to, to verify sources, authenticate reality and trace the provenance of content.
The team looked to establish an initial premise on the most effective interventions and how news organisations can collaborate on making them work and understood along with some best practice guidelines on ways working together.
The project complemented and leaned into the work of key industry initiatives of late – such as C2PA and Project Origin – to explore real-world, practical efforts to benchmark truly global collaborative efforts and the value of sharing insights around best practices.
The Objective
The team split the project work into three sections:
Provenance: which looked at how to prove the origins of content and the benefits for content creators, journalists and audiences as well as some of the existing provenance solutions and standards.
Note: HAND (Human & Digital) participated in this section.Detection: Which considered the tools that exist to flag manipulated material., demonstrating what can currently be achieved in both video and still images in both these areas.
It also considered where to use Provenance and where to use Detection tools in the chain.
Collaboration & Information sharing; This was mainly editorial where editorial teams collaborated worldwide across newsrooms to collaboratively share insights and intelligence, and where necessary help others to verify, authenticate or debunk stories and images/ videos. Uniquely, seven global newsrooms participated in the initiative in Mid-August 2024.
The Results: HAND's Contribution to the Solution
As part of the Provenance demonstration at IBC, HAND (Human & Digital), working with fellow participant company Eluvio, demonstrated an example of end-to-end provenance in a proof-of-concept workflow. The use case: In August, a fraudulent deepfake video of President Joe Biden’s Oval Office address (announcing his departure from the 2024 Presidential election) was shared online. Using the original legitimate PBS Television broadcast, HAND set about creating a unique, ISO-level identifier for President Joe Biden, and this resolvable ID was embedded as cryptographic metadata in the Eluvio Content Fabric system, supporting the Coalition for Content Provenance and Authenticity (C2PA) Content Credentials standard. The source attribution information about this video file was then detectable as being authentic vs. the deepfake version shared online, which was lacking C2PA Content Credentials metadata. With 64 elections around the world in 2024, and a historic U.S. election, much work remains to be done to address the disinformation crisis promulgated by the spread of malicious deepfakes.
2023
Accelerator Project:
Real-Time XR Sport Edge
Champions & Participants
Champions: Kings College London, Trinity College Dublin, Prima Terra Labs, Outernet Global, Vodafone Group, University of Southampton, RVPA, Bowie State University
Participants: D&B Solutions, Salsa Sound, Sparkup, HAND (Human & Digital), AMD, HearMeCheer, MOVRS
Official Project Showcase Page:
https://show.ibc.org/accelerator-project-real-time-xr-sport-edge
The Challenge
Continuing R&D from the award-winning 2021 Accelerator project that started to explore 5G Location-Based XR, this revived 2023 Accelerator aimed to build on the latest, cutting-edge innovations in interactive digital athletes, motion capture, AI and high-speed edge content delivery for live XR sports workflows for virtual 3D worlds, and the potential use cases into OTT platforms.
The original aim of this 2023 Challenge was to broadcast digital twin XR Sports using high-speed combat MMA (Mixed Martial Arts) athletes into live, immersive high-end graphics, spatial and social audio, and real-time, edge-compute deliverability to fans in a 3D virtual world via VR headsets, computers, mobile devices and also simultaneously to an outdoor Location-Based Experience (LBE) venue at The Outernet in London.
Pitched by Prima Terra Labs at IBC Kickstart Day, the Final POC Results were created by an all-star consortium of Champions and Participants with expertise in sports broadcasting, virtual production, 3D animation, spatial & social audio, edge engineering, academia, and talent identification.
The Objective: HAND's Contribution to the Solution
Digital Twins and Talent Identity for Athletes:
Prima Terra Labs captured the likeness of six fighters from the Surrey Martial Arts Club and 21st Century Combat, including a Pro K1 fighter, coach and gym owner, David Zetolofsky. Over the course of two days at the White Light/D&B Solutions studio in Reading, UK, the fighters were captured through the Live Link Face for Unreal Engine app and a combination of photos and videos so that the fighters’ digital twin MetaHumans could be created.
Their new MetaHuman images were then added to the HAND (Human & Digital) talent identity registry database, along with thumbnails of their photographic images for the purpose of providing reliable verification of the fighters as both notable real, legal and natural persons, and with their consent – as 3D scans of their digital replicas.
The Results
Core Outcomes:
- Real-time motion capture and broadcast of high-speed XR sports (Mixed Martial Arts) with two or more athletes
- Low-to-no latency connectivity and delivery via edge technologies
- Create and use virtual avatars of real-world athletes, augmented with VFX
- High-quality, photorealistic graphics in immersive environments
- Audiences in virtual 3D worlds will be able to participate and spectate on multiple devices (VR headsets, mobiles, computers)
- Athlete IP / NILV licensing consideration via interoperable talent ID labeling for provenance verification and automation.
Accelerator Project:
Synthetic Humans for Entertainment & Accessibility
Champions & Participants
Champions: RAI, EBU, ITV, VRT, Unity, YLE, BBC, Verizon Business, Kings College London, University of Southampton, Unreal Engine / Epic Games
Participants: Signly, Pluxbox, D&B Solutions, V-Nova, HAND (Human & Digital), 4DR Studios, Respeecher
Official Project Showcase Page: https://show.ibc.org/accelerator-project-synthetic-humans-entertainment-accessibility
The Challenge
The aim of this 2023 Accelerator was to present two different use cases based on synthetic humans: firstly, a theatre filled with the melodic tones of opera legend Maria Callas, and secondly, a photorealistic British sign-language interpreter Matt Kirby, to address important aspects of accessibility in broadcasting. These two distinct use cases aimed to demonstrate how synthetic humans can be used to captivate audiences in visually stunning, emotionally moving, and inclusive ways.
The Objective
Both Proof of Concept workstreams used a variety of production techniques, such as multiple motion capture and synthetic human creation technology toolkits, exploring how synthetic humans can accurately and realistically replicate real human movements, facial expressions and voice, to create and publish more believable, lifelike characters for both posthumous and living humans. Specific areas of R&D included:
The Results
The final POC results showcased how synthetic humans can increase inclusivity and allow endless storytelling possibilities within emerging platforms, and be used to enhance traditional media, such as television programs, live on-air presenting, and broadcasting. As part of the Final POC Results, the consortium also produced a technical paper PDF that captured the key learning points, specific industry calls to action, and suggested policy guidelines on AI and sustainable production in synthetic media and humans.
Site Links
Quick Links
join our Newsletter
Sign up for our newsletter for details on upcoming events, product launches, and updates.
Copyright © 2024 Human & Digital, Inc. (DBA HAND). All rights reserved. ®