CardioFAD Focus, Align, and Diffuse:
Time-Series-Aware Keyframe Diffusion
for Cardiac Dynamic Synthesis from Sparse Observations

TL;DR

CardioFAD turns a single cardiac observation into a dynamic sequence with motion-salient keyframes and time-series guidance.

From one cardiac observation to a dynamic sequence.

CMR

Volume2Seq
Observed input

Single 3D volume

Slice 7 / 13Frame 19 / 50

Synthesized keyframes
CMR keyframe 1 of 10: frame 3 of 50, slice 7 of 13
Keyframe 1 / 10 · Frame 3 / 50
Generated sequence
Ground truth

3D+t sequence

7 / 13

ECHO

Frame2Video
Observed input
Single observed echocardiography frame

Single frame

Synthesized keyframes
ECHO keyframe 1 of 8: frame 2 of 125
Keyframe 1 / 8 · Frame 2 / 125
Generated sequence
Ground truth

2D+t video

FIGURE 1Teaser
(a) Challenge: Sparse observations create temporal ambiguity, while ECG and visual signals may be misaligned. (b) Solution: Discover keyframe anchors that make motion recoverable and alignable. (c) Principle-to-Performance: Keyframes satisfying both principles improve synthesis quality.

Abstract

Dense cardiac dynamics are essential for assessing cardiac function, yet in practice clinicians often observe only extremely sparse measurements. However, recovering full dynamics from such sparsity is a highly ill-posed problem due to the lack of temporal cues. While auxiliary time-series signals like ECG provide physiological guidance, effectively exploiting them is challenging due to the cross-modal representation gap and phase misalignment. In this work, we present CardioFAD, a Focus-Align-Diffuse paradigm that reformulates sparse cardiac dynamic synthesis as a recoverability-alignability anchor discovery problem. Our core insight is that resolving extreme temporal ambiguity requires a compact set of anchor states that are both sufficient to recover the visual trajectory and reliable for ECG-visual alignment. We therefore learn motion-salient keyframes under two complementary principles: trajectory sufficiency, which preserves trajectory-critical intra-modal motion content, and anchor alignment, which promotes consistent inter-modal correspondence at these anchors. We implement the Focus and Align modules optimized by the two aspects. Diffuse then performs two-stage generation by first synthesizing keyframes and subsequently interpolating the remaining frames. Extensive experiments on cardiac MRI and echocardiography videos demonstrate superior visual quality, temporal coherence, and improved functional consistency under the extreme one-shot regime.

Key insight

The right keyframes make motion both recoverable and alignable.

Trajectory sufficiency

Preserve the motion needed to reconstruct the full sequence.

Anchor alignment

Connect visual states and time-series cues at informative phases.

A compact set of anchors. A coherent cardiac trajectory.

How CardioFAD works

Learn informative anchors. Align temporal cues. Generate cardiac dynamics.

Challenge One cardiac observation leaves the motion between frames unknown.

Reduced motion is enabled. The animation stays paused until you choose to play; method details are available below.

Full framework & method details
FIGURE 2Framework
Overview of CardioFAD. Focus learns trajectory-informative keyframes. Align establishes keyframe-weighted ECG–visual alignment. Diffuse first generates keyframes, then interpolates the remaining frames.

Focus: learn motion-salient anchors

Focus learns motion-salient keyframes from the time-series signal. Non-keyframe latents are masked, and a 3D UNet reconstructs the trajectory from the retained anchors. Reconstruction loss on the non-keyframe positions encourages the anchors to preserve essential cardiac dynamics.

Align: connect the modalities

Align matches visual and time-series features by cardiac phase. Its bidirectional contrastive objective emphasizes nearby phases and gives informative keyframes greater weight.

Diffuse: complete the dynamics

Diffuse first generates keyframes from the sparse observation and aligned time-series guidance. These generated anchor latents then condition the same Video DiT while it fills in the remaining positions; the anchors stay fixed during this second stage.

Results

One volume or slice for CMR. One frame for ECHO.

CMR synthesis

+1.90 dB

Higher PSNR

UK Biobank · Volume2Seq

32.41 vs. 30.51

Temporal quality

40.7%

Lower FVD

EchoNet-Dynamic

137.70 vs. 232.17

Cardiac functional consistency

4.17 vs. 4.98

EF MAE (lower is better)

EchoNet-Dynamic

EF estimated from generated videos

Quantitative Results

UK Biobank · Volume2Seq3D+t CMR synthesis

CMR Volume2Seq synthesis on UK Biobank. Higher PSNR and SSIM, and lower LPIPS, FID and FVD are better.
MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
VideoComposer28.150.8430.107128.761762.34
MotionCtrl28.460.8250.104119.521334.87
Tora29.740.8540.09297.301025.57
MotionPro27.880.8170.116145.391887.96
CardioFAD (One-stage)30.510.8610.08282.481007.63
CardioFAD (Ours)32.410.8850.06459.96736.01

UK Biobank · Slice2Seq3D+t CMR synthesis

CMR Slice2Seq synthesis on UK Biobank. Higher PSNR and SSIM, and lower LPIPS, FID and FVD are better.
MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
VideoComposer21.860.4970.289314.854574.61
MotionCtrl21.250.4880.294365.974914.27
Tora23.100.5390.251267.923109.75
MotionPro22.090.5010.269292.173756.61
CardioFAD (One-stage)23.750.5590.231211.492785.30
CardioFAD (Ours)25.600.5870.198163.121847.73

EchoNet-Dynamic2D+t ECHO synthesis

ECHO synthesis on EchoNet-Dynamic. Higher PSNR and SSIM, and lower LPIPS, FID and FVD are better.
MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
General Video Generation
VideoComposer22.780.5390.10948.25284.16
Tora23.900.5480.10252.06241.96
ECHO Video Generation
HeartBeat21.240.5270.12258.97376.51
EchoNet-Synthetic23.580.5400.10749.87267.35
ECHOPulse25.630.5680.08137.05196.28
CardioFAD (One-stage)25.140.5560.08946.82232.17
CardioFAD (Ours)27.730.5960.06726.92137.70

EchoNet-Pediatric (A4C)2D+t ECHO synthesis

ECHO synthesis on EchoNet-Pediatric (A4C). Higher PSNR and SSIM, and lower LPIPS, FID and FVD are better.
MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
General Video Generation
VideoComposer27.490.6330.08934.74346.31
Tora27.860.6380.08228.94361.09
ECHO Video Generation
HeartBeat26.710.6290.08837.56415.71
EchoNet-Synthetic26.080.6240.09239.93408.12
ECHOPulse28.140.6410.08029.63357.74
CardioFAD (One-stage)28.320.6430.07829.23339.15
CardioFAD (Ours)29.480.6570.07323.21286.29

EchoNet-Pediatric (PSAX)2D+t ECHO synthesis

ECHO synthesis on EchoNet-Pediatric (PSAX). Higher PSNR and SSIM, and lower LPIPS, FID and FVD are better.
MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓
General Video Generation
VideoComposer27.260.6210.08442.19389.07
Tora28.370.6380.08135.52347.21
ECHO Video Generation
HeartBeat27.650.6300.07945.02407.26
EchoNet-Synthetic26.230.6120.09048.82432.50
ECHOPulse28.500.6390.08037.57367.46
CardioFAD (One-stage)28.770.6400.07939.17381.06
CardioFAD (Ours)29.590.6450.07127.12293.86

Scroll horizontally to view all metrics

Qualitative Results

Video Results

Observed input Keyframes Current frame

CMR Video Generation Examples

Volume2Seq

Single volume full 3D+t CMR sequence

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Slice2Seq

Single slice full 3D+t CMR sequence

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

ECHO Video Generation Examples

Frame2Video

Single frame full 2D+t ECHO video

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

Ground truth

Video to be added

CardioFAD

Video to be added

BibTeX