UltraGuard: Ultrasound-Assisted Biometric Watermarking Against Face Video Deepfakes
Ultrasound-Assisted Biometric Watermarking Against Face Video Deepfakes
The authors propose UltraGuard, which fuses smartphone ultrasonic sensing with facial video to generate a dynamic watermark; the authors report average accuracy of 99.35% on source-side attacks and 100% on the propagation side.
The attacker understands the UltraGuard architecture and can obtain published facial videos, watermarked videos, and the static detector implementation, and can carry out replay, face swapping, video editing, watermark reuse, and adaptive attacks that construct synchronized forged videos and ultrasound signals; however, the attacker cannot tamper with the trusted sensing pipeline, inject forged sensor data, or obtain the target user's biometric–ultrasound correspondence, and dynamic Face Key verification is performed only in the trusted cloud.
- Existing video watermarks are mostly offline static schemes; they cannot verify whether a video comes from a real online user and are easily bypassed by replay attacks and private-model forgery; forgery detection and face authentication each cover only the propagation or the acquisition end, lacking end-to-end protection.
- UltraGuard uses the phone's earpiece speaker to emit ultrasound and the microphone to collect facial biometric features, cross-modally aligns and fuses them with ArcFace video features, generates a static User Key and a content-related dynamic Face Key, embeds them into the lip ROI via InvisMark, and verifies by cloud-side comparison.
- On a dataset of 50 volunteers and four phone models, the authors report 96.46% accuracy on legitimate samples, an average of 99.35% for source-end attacks, and 100% for both watermark reuse and editing attacks at the distribution end; the lowest end-to-end latency on three phone models is 142.40–213.64 ms.
- The authors point out that the effective ultrasonic sensing distance is limited, head movement and occlusion may cause authentication failure, the sensing pipeline is at risk if the hardware is compromised, video compression may impair the watermark, and dataset diversity and attack-scenario coverage are insufficient; the experiments cover only 50 volunteers and 6 types of attack scenarios.
- Evaluated models
- UltraGuard
- Benchmarks
- 自建同步视频-超声数据集(50 名志愿者,约 25 小时合法样本、超过 135 小时攻击样本)
- Metrics
- AccuracyTPRFPRTNRFNREERAUCHamming distance
The study proposes UltraGuard, a face video watermarking method that uses a phone's built-in speaker and microphone to emit ultrasound, collect biometric features, and fuse spatiotemporal dynamic features of the video, enabling real-time embedding during transmission. Experiments show it can resist forgery, editing and other attacks at the source and during propagation; in the future it could be deployed on mobile devices to provide real-time protection for facial videos.
Paper analysis
What problem does this paper attempt to solve?
Existing video watermarks cannot verify the liveness provenance of online real-time video; the authors propose a dynamic biometric watermark system from the source end to the propagation end.
Core problem: how to provide authenticity protection for online real-time facial videos across the entire process from the collection source to distribution, resisting deepfake forgery, replay, and editing attacks.
- Scenario and importance: Video conferencing and real-time calls are increasingly used in scenarios such as legal archiving, remote justice, and telemedicine; the authors believe forged facial videos generated by AIGC models such as Sora and Seedance are increasingly hard to distinguish, posing a threat to user privacy and security.
- Shortcomings of existing methods: The authors point out that the mainstream approach is to embed watermarks into forged videos, which are easily removed by attackers or bypassed by privately trained generative models; existing video watermarks are mostly offline static schemes that cannot verify whether a video comes from a live real source, and once the watermark extractor is public it can be reverse-engineered; forgery detection only performs post-hoc identification, and face authentication only verifies identity at collection time, so neither covers the entire process.
- Core observation: The authors' key view is that source-end online dynamic watermarking is more secure by design than adding watermarks after generation—only videos of authenticated live users are given valid watermarks, and no watermark or an incorrect watermark is judged as forgery.
- Proposed scheme: UltraGuard uses the smartphone's built-in earpiece speaker and microphone to emit and collect ultrasound signals to extract biometric features, fuses them with spatiotemporal dynamic features of the video content and encodes them as a watermark, embedding it into the video in real time; the authors claim it is the first real-time online dynamic secure watermarking system.
- Threat model (Section 4.2):
- Attacker goals: forge authenticity evidence at the source end (replay, face swapping), or tamper with video content at the distribution end and pass verification.
- Attacker knowledge: understands the general working principle of the UltraGuard architecture and can obtain published facial videos, corresponding watermarked videos, and the static detector implementation; the cloud verification material for the dynamic Face Key is not public.
- Attacker capabilities: can carry out digital manipulations such as replay, face swapping, video editing, and watermark reuse, including adaptive attackers who know the multimodal consistency constraints and attempt to construct synchronized forged videos and ultrasound signals.
- Boundaries and assumptions: the user's phone (microphone, earpiece, front camera, operating system, UltraGuard components) is trusted and not compromised; the attacker cannot inject forged sensor data or obtain the target user's biometric–ultrasound paired observations; the registration step is trusted; the cloud verification service is trusted.
What related research exists?
Related work is divided into three groups: video watermarking, forgery detection, and phone authentication; the authors state that each of the three covers only part of the pipeline.
Video watermarking
- Traditional methods: spatial-domain [45], transform-domain [22], and temporal-domain [37] watermarking—trading off robustness against imperceptibility; the authors state that each has limitations in robustness or video quality.
- Deep learning methods [10, 15, 16, 42]: end-to-end joint optimization of watermark embedding and extraction, more robust to complex distortions such as transcoding and scaling. The authors point out that these techniques are still mainly oriented toward offline static watermarking, cannot verify the authenticity of video provenance, and do not guarantee sensitivity to forged tampering during propagation.
Video forgery detection
- Methods based on biological signals and physical consistency [23, 43]: exploiting physical priors such as abnormal expressions, unnatural blinking, and lighting and reflection inconsistencies; the authors state that these are effective against early forgeries but degrade as generative models advance.
- Data-driven methods [31, 38] and detection targeting diffusion-model "fingerprints" [6]: training CNNs or ViTs on large-scale real/fake datasets to capture pixel, frequency-domain, and noise artifacts. The authors point out that existing detectors have limited generalization ability and degrade when facing unseen forgery methods.
Smartphone-based human authentication
- 3D/infrared [33, 49] and iris recognition [7]: the authors state that these suffer from high cost, lighting constraints, or intrusiveness.
- Ultrasonic schemes [4, 26]: using built-in speakers/microphones, low-cost, unaffected by lighting, and non-intrusive; the authors point out that pure authentication only distinguishes users, carries no security information, and lacks continuity.
Baselines and comparison targets: the paper does not implement baseline methods for numerical comparison; Table 1 presents a feature comparison table listing three categories of methods (video forgery detection [6, 31, 38], traditional video watermarking [10, 16, 45], face authentication [4, 36, 49]) across six items: source-side liveness, propagation protection, dynamic content binding, edit resistance, replay resistance, and online deployment; UltraGuard is ✓ on all six (replay resistance marked as Strong).
The authors position this paper as: fusing ultrasonic biometric authentication with video watermarking to generate a dynamic, face-bound biometric watermark, a scheme covering end-to-end security from generation to propagation, distinct from all the above methods that cover only a single stage (Section 2.4).
How does the paper address this problem?
Ultrasonic and video bimodal features are aligned and fused to generate a User Key + dynamic Face Key watermark embedded in the lip ROI, verified by cloud-side comparison.
Overall approach: only face videos carrying the correct dynamic watermark are treated as genuine. The system uses ultrasound signals to provide liveness biometric evidence and uses video content to generate content-dependent dynamic codes; the two are fused and embedded into the video via InvisMark, and the detection side regenerates the watermark and compares. UltraGuard adopts a late-fusion strategy, for which the authors give the reasons of making it easy to leverage a pretrained video encoder and avoiding the information-richer video modality suppressing training of the ultrasound modality.
Ultrasound sensing and signal processing (Section 3)
- Hardware and signals: the earpiece speaker emits a continuous wave (CW) with 16 orthogonal subcarriers at frequencies 17.15–22.4 kHz with 350 Hz spacing; the microphone records at 48 kHz, and the front camera captures synchronously at 30 FPS.
- Demodulation: each subcarrier undergoes digital coherent demodulation, 50 Hz low-pass filtering, and downsampling by D=16, yielding a complex baseband signal Bk(t) = Ik(t) + jQk(t) at an effective sampling rate of 3 kHz, i.e., the in-phase/quadrature motion component of each subcarrier.
- Synchrony verification: Fig. 2 shows that the I/Q component fluctuations of the 17.15 kHz subcarrier align with mouth movements in the video on a shared time axis, on which basis the authors conclude that ultrasound can reliably capture articulatory kinematic details.
Ultrasound encoder Eu (Section 6.1)
- Goal: project ultrasound features of different users into a separable high-dimensional space, so that even a single sample from a newly enrolled user forms a distinct cluster, supporting source-side anti-spoofing.
- Structure: a symmetric encoder-decoder, with four cascaded multi-scale convolution blocks (MSCB) + channel attention (CA), a position-encoded attention block outputting 512-dimensional latent features; GAP + a projection layer generates downstream features; the decoder has four transposed convolution blocks to reconstruct the waveform.
- Training: an MSE reconstruction loss Lm paired with a weak decoder Du (the authors say this adversarial design preserves intra-class variation and avoids intra-class feature collapse); simultaneously a cosine similarity loss Lc = 1 − (a⊤ b)/(|a|2 · |b|2) aligns the Eu output to the feature space of a frozen pretrained video encoder Ev, with total loss Ls1 = λc·Lc + λm·Lm, which the authors say gives the ultrasound feature space generalization ability beyond the limited users collected.
Fused features and user verification (Section 6.2)
- Fusion module: the two feature streams pass through a dimension transformation module (DTM, two fully connected layers + GELU, outputting 256 dimensions), 8-head cross-modal cross-attention, and three "linear + BN + GELU" blocks to obtain fused features; metric learning uses Triplet Loss Lt = max(|xa − xp|22 − |xa − xn|22 + α, 0) with margin α = 1.0.
- Verification logic (Algorithm 1): first require the video feature similarity to exceed threshold βv, then judge whether the ultrasound and fused features respectively exceed βu and βf; all three thresholds are set to 0.5.
- Dynamic encoding (Algorithm 2): at enrollment a static User Key is assigned (U = 32 bits); the real-time video feature v minus the enrollment feature v0, after normalization, is multiplied with the enrolled normalized fused feature f0, and binarized via tanh and sign to obtain the dynamic Face Key (F = 224 bits), concatenated into a 256-bit watermark code. The authors say the encoding undergoes lossy compression and fused-vector transformation, making it hard to reverse.
Video encoder and watermark embedding (Sections 6.3, 6.4)
- Video encoder Ev: pretrained ArcFace, extracting a 512-dimensional identity embedding per frame and then aggregating into a video-level representation; the authors consider that the property of face feature space—different people forming clusters and the same person with different expressions spreading within a cluster—happens to satisfy both the need for source-side user discrimination and the need for the dynamic watermark to distinguish different content from the same person (Fig. 5).
- Embedding and detection: the InvisMark end-to-end neural watermarking framework embeds the 256-bit payload, with training including a differentiable distortion layer (compression, scaling, filtering, geometric transformation); simple scenarios verify statically locally with the User Key, while high-security scenarios upload to the cloud to regenerate the Face Key and compare.
Edge deployment (Section 6.5)
- Key-region localization: SCRFD-500M detects the face and mouth-corner keypoints, generating a 256×256 lip ROI, and the watermark is embedded only in the ROI; when no face is detected, the system stops emitting ultrasound and generating watermarks.
- Lightweighting: the edge side uses a lightweight encoder with a MobileFaceNet backbone + ArcFace loss; a sliding-window cache pool uniformly takes K=5 keyframes within a 1-second window (30 FPS) to embed the watermark, reducing the watermark frequency by about 83.3%; the remaining modules undergo INT8 quantization (QAT). Table 2 shows total parameters drop from 242.49 MB to 42.41 MB, and the authors say overall computation and memory usage are reduced by more than 82.51%.
What experiments did the paper conduct?
Self-built 50-person video-ultrasound dataset; average accuracy 99.35% on source-side attacks and 100% on the propagation side; modality, time window, distance, angle, phone model, and latency were also measured.
Experimental setup
- Data: a self-built synchronized video-ultrasound dataset, 50 volunteers (25 male, 25 female), 10 randomly selected for testing, 25–30 minutes per person, about 25 hours of legitimate samples in total; attack samples exceed 135 hours (S1 1 hour, S2 125 hours, S3 0.5 hours, S4 1 hour, P1 about 7 hours, P2 about 0.5 hours). Time window T = 5 seconds, collection distance 20–25 cm (Sections 7.1, 7.2).
- Devices: collected with four phones: Samsung S10, Samsung S22 Ultra, Vivo X200 Pro, and OnePlus 15; latency tests used Samsung S10, vivo iQOO8, and OnePlus 15T, all running on CPU.
- Attack construction: S4 uses the face-swap model DynamicFace [40] to replace the face in legitimate videos; P2 uses Edit-Yourself [11] to alter mouth movements so that the semantics are completely different while retaining the original watermark.
- Metrics: TPR, FPR, TNR, FNR, EER, AUC, Accuracy; the propagation side uses Hamming distance to measure watermark similarity (Section 8.1).
- Protocol: average ROC computed over 20 iterations of randomly selected enrollment samples; QAT impact assessed with 100 independent embed-detect trials; latency and power averaged over 100 test samples.
- Training: RTX 4090; Eu stage lr 5e-4, 400 epochs, batch 512, λc=λm=1; fusion stage lr 5e-4, batch 256; QAT stage lr 1e-5, 200 epochs, batch 16, QNNPACK backend.
Main results
The paper's main results are given mainly in figures (Fig. 8, Fig. 9) and in numbers in the text; no per-scenario numerical table is provided. The table below summarizes the numbers stated in the text and figure captions.
Swipe horizontally to read the table
Evaluated object Metric Value Source Legitimate samples Accuracy 96.46% Fig. 9 left Source-side attacks S1–S4 Average Accuracy 99.35% Fig. 9 middle, Section 8.3 Propagation side P1 (watermark reuse) Accuracy 100% Section 8.4 Propagation side P2 (editing) Negative-sample detection Accuracy 100% Section 8.4 Overall (average of 20 runs) AUC / EER 0.9742 ± 0.0029 / 0.0225 Fig. 8 P2 edited samples Average watermark similarity 82.33 ± 5.31% Section 8.4 - The authors' interpretation of the source-side results: S1/S2 are identified due to cross-modal consistency constraints; in S3, screen specular reflection versus skin diffuse reflection and the acoustic impedance difference between glass and skin can be captured by the 16-subcarrier design; S4 face swapping changes facial content while the ultrasound remains bound to the original facial biometric structure, producing a cross-modal mismatch.
- The authors' interpretation of P2: the Face Key comes from the overall representation of facial content within time window T, and any local modification produces a different encoding; the fixed User Key remains unchanged under such attacks, and the dynamic Face Key provides a second layer of protection.
- The abstract and contributions give the summary statement "average 99.35% and 100% accuracy", consistent with Sections 8.3 and 8.4; the paper does not report separate accuracy values for each attack scenario.
Impact of quantization (QAT) on watermark detection
- What was measured: INT8 quantization of the edge-side watermark embedding module, comparing detection accuracy of direct quantization versus QAT (100 independent trials; the detection module is not quantized on the server side).
- Results: detection accuracy rose from 87.01 ± 6.31% to 95.48 ± 2.12% (Section 8.2).
- Authors' interpretation: QAT effectively mitigates the accuracy degradation caused by quantization, making on-device embedding reliable.
System latency and power consumption
- What was measured: end-to-end latency and power consumption of three phones (Samsung S10, vivo iQOO8, OnePlus 15T) under simulated real-time video calls, running on CPU, K=5, embedding only in the lip ROI, averaged over 100 samples (Fig. 15, Fig. 16).
- Results: the lowest end-to-end latencies were 213.64 ms, 168.91 ms, and 142.40 ms respectively; sustainable operation for 2.89, 6.29, and 6.28 hours.
- Authors' interpretation: more than 5 keyframes can be watermarked within 1 second, meeting the processing requirements of real-time video calls, and on-device deployment is feasible.
Other ablations and analyses
- Modality combinations (Fig. 10): joint decision of video + ultrasound + fusion 96.46%, video only 94.20%, ultrasound only 89.14%, simple concatenation 93.87%.
- Time window T (Fig. 11): average accuracy 96.48% across different window lengths, 96.17% at T = 1 s.
- Distance (Fig. 12): overall accuracy at 10/15/20/25 cm all above 87.77%, average 93.62%; at 10 cm it drops to 87.77%, which the authors attribute to the close-range camera capturing only part of the face.
- Angle (Fig. 13): accuracy within about ±20 degrees all above 93.89%.
- Phone models (Fig. 14): accuracy on all four phones above 95.50%, average 96.42%.
- The paper does not report statistical significance tests; the above factor experiments all evaluate only legitimate-sample accuracy (Section 8.5).
What are some points that can be further explored?
The authors list limitations in four aspects: sensing distance, hardware trustworthiness, compression robustness, and data diversity; the experiments cover 50 people and 6 types of attacks.
Limitations and future directions noted by the authors
- Sensing distance and occlusion (Discussion, first paragraph): ultrasonic echoes limit the effective sensing distance; when the user is far from the phone, the weakened echo may reduce authentication reliability; large head movements and frequent facial occlusion may disrupt the temporal consistency between ultrasonic and video features, causing authentication failure; the authors state that complementary sensing modalities could be introduced in the future to improve robustness under dynamic conditions.
- Trustworthiness of the sensing pipeline (Discussion, second paragraph): if the hardware is compromised, unauthorized access to internal sensor data or injection of forged data into the acquisition pipeline remain potential risks; the authors state that hardware-level encryption and data-flow integrity verification could be integrated in the future.
- Video compression (Discussion, third paragraph): video compression during transmission may impair watermark integrity and affect subsequent verification; the authors state that watermark embedding could in the future be more tightly integrated with the video coding pipeline, directly accounting for codec distortion.
- Data and attack coverage (Discussion, fourth paragraph): the current dataset does not adequately represent the diversity of users, demographic groups and environmental conditions in real deployments, and the evaluated attack scenarios do not cover all possible threats; the authors state that future work should expand data diversity and study more attack strategies.
- Threat model boundaries (Section 4.2.2): direct physical attacks on trusted sensors, malicious modification of sensor drivers/firmware, and privileged operating-system compromise are outside the scope of this paper; the authors state that the trusted sensing pipeline could be strengthened in the future through hardware-level encryption and data-flow integrity verification.
Scope of experimental coverage
- The evaluated subject is the authors' own UltraGuard system; no numerical baseline comparison with other watermarking or detection methods is made, and only Table 1 gives a feature-level comparison.
- Data come from 50 volunteers (25 male, 25 female), of whom 10 were randomly selected for testing; the collection devices were four phones: Samsung S10, Samsung S22 Ultra, Vivo X200 Pro, and OnePlus 15 (Section 7.1).
- Attack evaluation covers 4 source-side scenarios (S1 simple replay, S2 random pairing, S3 advanced replay, S4 face-swap forgery) and 2 propagation-side scenarios (P1 watermark reuse, P2 editing); face swapping uses DynamicFace and editing uses Edit-Yourself (Section 7.2).
- Latency and power consumption were measured on the CPUs of three phones: Samsung S10, vivo iQOO8, and OnePlus 15T, averaged over 100 samples each; the overall ROC is averaged over 20 random enrollment samples (Sections 8.2, 8.6).
- The paper does not report separate accuracy values for each attack scenario, does not report statistical significance tests, and does not report human evaluation or human-perception discrimination experiments.
Summarize the main content of the paper
UltraGuard fuses ultrasound biometric features with facial video into a dynamic watermark, and the authors report defense accuracy of 99.35% at the source end and 100% at the distribution end.
A defense paper in the mobile sensing direction: it provides anti-counterfeiting watermarks for online real-time facial videos covering the entire process from the collection source to distribution, countering deepfake forgery, replay, and editing.
- Problem: The authors consider existing video watermarks to be offline static schemes that cannot verify whether a video comes from a live real user and are easily bypassed by replay attacks and watermark reverse engineering; forgery detection and face authentication each handle only one part, with no end-to-end protection.
- Method: UltraGuard uses the phone earpiece to emit 16-subcarrier ultrasound (17.15–22.4 kHz) and the microphone to collect facial biometric features, fusing them with ArcFace video features via cross-modal alignment (cosine loss) and Triplet Loss; the watermark consists of a 32-bit static User Key and a 224-bit content-related dynamic Face Key, embedded into the lip ROI using InvisMark; the detection end regenerates the watermark for comparison, and a mismatch is judged as tampering. On the edge side, it is deployed via SCRFD detection, K=5 sparse frame embedding, and INT8 QAT quantization.
- Main results: On a self-built dataset of 50 volunteers and four phone models, legitimate sample accuracy is 96.46%, with an average AUC of 0.9742 ± 0.0029 (Fig. 8, Fig. 9); the average accuracy for the four types of source-end attacks is 99.35% (Fig. 9); the detection accuracy for both watermark reuse and editing attacks at the distribution end is 100%, and the average watermark similarity of edited samples is 82.33 ± 5.31% (Section 8.4); QAT raises the detection accuracy after on-device embedding from 87.01 ± 6.31% to 95.48 ± 2.12% (Section 8.2); the lowest end-to-end latency on three phone CPUs is 142.40–213.64 ms (Fig. 15).
- Authors' conclusion: The authors claim the system is the first real-time online dynamic secure watermarking system and can be deployed on ordinary phones to provide real-time protection for facial videos; they also acknowledge limitations in sensing distance, occlusion, hardware trustworthiness, compression robustness, and data diversity, and plan to introduce complementary modalities, hardware-level protection, and expand data and attack coverage in the future.