runtime 85 minutes

[thundercarrot]

The example command in the Lyra-2 readme takes about 85 minutes on an H200. Does that sound right?

[frankshen07]

We haven’t tested it on H200, but it should be faster than H100. Does the log show any errors or progress?

[thundercarrot]

It seems to spend a lot of time in

[05-01 07:36:34|INFO|lyra_2/_src/models/lyra2_model.py:2244:_prepare_lyra2_inputs] Buffer hist padding: ['pos0:f1k=zeros', 'pos1:f4s=spatial_coord', 'pos2:f1s=spatial_coord', 'pos3:f16k=zeros', 'pos4:f2k=zeros', 'pos5:f1k=zeros'], C_buf=1280, _max_spatial=5

Here's the whole run:

~/lyra/Lyra-2$ time PYTHONPATH=. python -m lyra_2._src.inference.lyra2_zoomgs_inference   --input_image_path assets/samples   --sample_id 4   --e
xperiment lyra2   --checkpoint_dir checkpoints/model   --prompt_dir assets/samples   --output_p
ath outputs/zoomgs   --num_frames_zoom_in 81   --num_frames_zoom_out 241   --zoom_in_strength 0
.5   --zoom_out_strength 1.5
[05-01 07:35:08|INFO|lyra_2/_ext/imaginaire/checkpointer/dcp.py:195:<module>] for the back comptiable pytorch! New DefaultLoadPlanner class is created.
[05-01 07:35:08|INFO|lyra_2/_ext/imaginaire/utils/misc.py:150:set_random_seed] Using random seed 1.
[05-01 07:35:08|INFO|lyra_2/_ext/imaginaire/utils/config_helper.py:187:import_all_modules_from_package] Reloading all modules from package lyra_2._src.configs
[05-01 07:35:09|INFO|lyra_2/_ext/imaginaire/utils/misc.py:150:set_random_seed] Using random seed 0.
[05-01 07:35:09|WARNING|lyra_2/_src/models/wan_t2v_model.py:149:__init__] DiffusionModel: precision torch.bfloat16
[05-01 07:35:09|WARNING|lyra_2/_src/models/wan_t2v_model.py:150:__init__] Flow Matching: precision torch.float32
[05-01 07:35:09|INFO|lyra_2/_src/tokenizers/wan2pt1.py:663:_video_vae] loading ./checkpoints/vae/vae.pth
[05-01 07:35:09|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on DiffusionModel: set_up_tokenizer: 0.6380 seconds
[05-01 07:35:10|INFO|lyra_2/_src/modules/clip.py:482:load_model_torch] loading weights from ./checkpoints/image_encoder/model.pth
[05-01 07:35:13|INFO|lyra_2/_src/modules/conditioner.py:353:__init__] Initialized embedder #0-text: 
 TextAttrEmptyStringDrop 
        input key: ['t5_text_embeddings']
        Param count: 0 
        Trainable: None
        Dropout rate: 0.2
        Output key: [crossattn_emb]
[05-01 07:35:13|INFO|lyra_2/_src/modules/conditioner.py:353:__init__] Initialized embedder #1-fps: 
 ReMapkey 
        input key: fps
        Param count: 0 
        Trainable: None
        Dropout rate: 0.0
        Output key: fps 
        Dtype: None
[05-01 07:35:13|INFO|lyra_2/_src/modules/conditioner.py:353:__init__] Initialized embedder #2-padding_mask: 
 ReMapkey 
        input key: padding_mask
        Param count: 0 
        Trainable: None
        Dropout rate: 0.0
        Output key: padding_mask 
        Dtype: None
[05-01 07:35:13|INFO|lyra_2/_src/modules/conditioner.py:353:__init__] Initialized embedder #3-wanclip: 
 Wan2pt1CLIPEmbLyra2 
        input key: ['last_hist_frame', 'video', 'i2v_WAN2PT1_cond_latents', 'cond_latent_mask', 'cond_latent_buffer']
        Param count: 0 
        Trainable: None
        Dropout rate: 0.0

[05-01 07:35:13|INFO|lyra_2/_src/models/lyra2_model.py:217:build_net] Constructing clean patch embeddings before FSDP
[05-01 07:35:14|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:401:init_clean_patch_embeddings] Constructed 6 clean patch embedding layers (weights not yet copied).
[05-01 07:35:14|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on meta to cuda and broadcast model states: 0.3550 seconds
[05-01 07:35:14|INFO|lyra_2/_src/models/lyra2_model.py:296:build_net] Freezing model and unfreezing Lyra2AttentionBlock layers matching: ['cam_encoder', 'buffer_encoder', 'self_attn', 'clean_patch_embeddings', 'patch_embedding']
[05-01 07:35:14|INFO|lyra_2/_src/models/lyra2_model.py:328:build_net] Enabled gradients for 534 parameters: ['blocks.0.buffer_encoder.0.weight', 'blocks.0.buffer_encoder.1.weight', 'blocks.0.cam_encoder.weight', 'blocks.0.self_attn.k.bias', 'blocks.0.self_attn.k.weight', 'blocks.0.self_attn.norm_k.weight', 'blocks.0.self_attn.norm_q.weight', 'blocks.0.self_attn.o.bias', 'blocks.0.self_attn.o.weight', 'blocks.0.self_attn.q.bias', 'blocks.0.self_attn.q.weight', 'blocks.0.self_attn.v.bias', 'blocks.0.self_attn.v.weight', 'blocks.1.buffer_encoder.0.weight', 'blocks.1.buffer_encoder.1.weight', 'blocks.1.cam_encoder.weight', 'blocks.1.self_attn.k.bias', 'blocks.1.self_attn.k.weight', 'blocks.1.self_attn.norm_k.weight', 'blocks.1.self_attn.norm_q.weight', 'blocks.1.self_attn.o.bias', 'blocks.1.self_attn.o.weight', 'blocks.1.self_attn.q.bias', 'blocks.1.self_attn.q.weight', 'blocks.1.self_attn.v.bias', 'blocks.1.self_attn.v.weight', 'blocks.10.buffer_encoder.0.weight', 'blocks.10.buffer_encoder.1.weight', 'blocks.10.cam_encoder.weight', 'blocks.10.self_attn.k.bias', 'blocks.10.self_attn.k.weight', 'blocks.10.self_attn.norm_k.weight', 'blocks.10.self_attn.norm_q.weight', 'blocks.10.self_attn.o.bias', 'blocks.10.self_attn.o.weight', 'blocks.10.self_attn.q.bias', 'blocks.10.self_attn.q.weight', 'blocks.10.self_attn.v.bias', 'blocks.10.self_attn.v.weight', 'blocks.11.buffer_encoder.0.weight', 'blocks.11.buffer_encoder.1.weight', 'blocks.11.cam_encoder.weight', 'blocks.11.self_attn.k.bias', 'blocks.11.self_attn.k.weight', 'blocks.11.self_attn.norm_k.weight', 'blocks.11.self_attn.norm_q.weight', 'blocks.11.self_attn.o.bias', 'blocks.11.self_attn.o.weight', 'blocks.11.self_attn.q.bias', 'blocks.11.self_attn.q.weight', 'blocks.11.self_attn.v.bias', 'blocks.11.self_attn.v.weight', 'blocks.12.buffer_encoder.0.weight', 'blocks.12.buffer_encoder.1.weight', 'blocks.12.cam_encoder.weight', 'blocks.12.self_attn.k.bias', 'blocks.12.self_attn.k.weight', 'blocks.12.self_attn.norm_k.weight', 'blocks.12.self_attn.norm_q.weight', 'blocks.12.self_attn.o.bias', 'blocks.12.self_attn.o.weight', 'blocks.12.self_attn.q.bias', 'blocks.12.self_attn.q.weight', 'blocks.12.self_attn.v.bias', 'blocks.12.self_attn.v.weight', 'blocks.13.buffer_encoder.0.weight', 'blocks.13.buffer_encoder.1.weight', 'blocks.13.cam_encoder.weight', 'blocks.13.self_attn.k.bias', 'blocks.13.self_attn.k.weight', 'blocks.13.self_attn.norm_k.weight', 'blocks.13.self_attn.norm_q.weight', 'blocks.13.self_attn.o.bias', 'blocks.13.self_attn.o.weight', 'blocks.13.self_attn.q.bias', 'blocks.13.self_attn.q.weight', 'blocks.13.self_attn.v.bias', 'blocks.13.self_attn.v.weight', 'blocks.14.buffer_encoder.0.weight', 'blocks.14.buffer_encoder.1.weight', 'blocks.14.cam_encoder.weight', 'blocks.14.self_attn.k.bias', 'blocks.14.self_attn.k.weight', 'blocks.14.self_attn.norm_k.weight', 'blocks.14.self_attn.norm_q.weight', 'blocks.14.self_attn.o.bias', 'blocks.14.self_attn.o.weight', 'blocks.14.self_attn.q.bias', 'blocks.14.self_attn.q.weight', 'blocks.14.self_attn.v.bias', 'blocks.14.self_attn.v.weight', 'blocks.15.buffer_encoder.0.weight', 'blocks.15.buffer_encoder.1.weight', 'blocks.15.cam_encoder.weight', 'blocks.15.self_attn.k.bias', 'blocks.15.self_attn.k.weight', 'blocks.15.self_attn.norm_k.weight', 'blocks.15.self_attn.norm_q.weight', 'blocks.15.self_attn.o.bias', 'blocks.15.self_attn.o.weight', 'blocks.15.self_attn.q.bias', 'blocks.15.self_attn.q.weight', 'blocks.15.self_attn.v.bias', 'blocks.15.self_attn.v.weight', 'blocks.16.buffer_encoder.0.weight', 'blocks.16.buffer_encoder.1.weight', 'blocks.16.cam_encoder.weight', 'blocks.16.self_attn.k.bias', 'blocks.16.self_attn.k.weight', 'blocks.16.self_attn.norm_k.weight', 'blocks.16.self_attn.norm_q.weight', 'blocks.16.self_attn.o.bias', 'blocks.16.self_attn.o.weight', 'blocks.16.self_attn.q.bias', 'blocks.16.self_attn.q.weight', 'blocks.16.self_attn.v.bias', 'blocks.16.self_attn.v.weight', 'blocks.17.buffer_encoder.0.weight', 'blocks.17.buffer_encoder.1.weight', 'blocks.17.cam_encoder.weight', 'blocks.17.self_attn.k.bias', 'blocks.17.self_attn.k.weight', 'blocks.17.self_attn.norm_k.weight', 'blocks.17.self_attn.norm_q.weight', 'blocks.17.self_attn.o.bias', 'blocks.17.self_attn.o.weight', 'blocks.17.self_attn.q.bias', 'blocks.17.self_attn.q.weight', 'blocks.17.self_attn.v.bias', 'blocks.17.self_attn.v.weight', 'blocks.18.buffer_encoder.0.weight', 'blocks.18.buffer_encoder.1.weight', 'blocks.18.cam_encoder.weight', 'blocks.18.self_attn.k.bias', 'blocks.18.self_attn.k.weight', 'blocks.18.self_attn.norm_k.weight', 'blocks.18.self_attn.norm_q.weight', 'blocks.18.self_attn.o.bias', 'blocks.18.self_attn.o.weight', 'blocks.18.self_attn.q.bias', 'blocks.18.self_attn.q.weight', 'blocks.18.self_attn.v.bias', 'blocks.18.self_attn.v.weight', 'blocks.19.buffer_encoder.0.weight', 'blocks.19.buffer_encoder.1.weight', 'blocks.19.cam_encoder.weight', 'blocks.19.self_attn.k.bias', 'blocks.19.self_attn.k.weight', 'blocks.19.self_attn.norm_k.weight', 'blocks.19.self_attn.norm_q.weight', 'blocks.19.self_attn.o.bias', 'blocks.19.self_attn.o.weight', 'blocks.19.self_attn.q.bias', 'blocks.19.self_attn.q.weight', 'blocks.19.self_attn.v.bias', 'blocks.19.self_attn.v.weight', 'blocks.2.buffer_encoder.0.weight', 'blocks.2.buffer_encoder.1.weight', 'blocks.2.cam_encoder.weight', 'blocks.2.self_attn.k.bias', 'blocks.2.self_attn.k.weight', 'blocks.2.self_attn.norm_k.weight', 'blocks.2.self_attn.norm_q.weight', 'blocks.2.self_attn.o.bias', 'blocks.2.self_attn.o.weight', 'blocks.2.self_attn.q.bias', 'blocks.2.self_attn.q.weight', 'blocks.2.self_attn.v.bias', 'blocks.2.self_attn.v.weight', 'blocks.20.buffer_encoder.0.weight', 'blocks.20.buffer_encoder.1.weight', 'blocks.20.cam_encoder.weight', 'blocks.20.self_attn.k.bias', 'blocks.20.self_attn.k.weight', 'blocks.20.self_attn.norm_k.weight', 'blocks.20.self_attn.norm_q.weight', 'blocks.20.self_attn.o.bias', 'blocks.20.self_attn.o.weight', 'blocks.20.self_attn.q.bias', 'blocks.20.self_attn.q.weight', 'blocks.20.self_attn.v.bias', 'blocks.20.self_attn.v.weight', 'blocks.21.buffer_encoder.0.weight', 'blocks.21.buffer_encoder.1.weight', 'blocks.21.cam_encoder.weight', 'blocks.21.self_attn.k.bias', 'blocks.21.self_attn.k.weight', 'blocks.21.self_attn.norm_k.weight', 'blocks.21.self_attn.norm_q.weight', 'blocks.21.self_attn.o.bias', 'blocks.21.self_attn.o.weight', 'blocks.21.self_attn.q.bias', 'blocks.21.self_attn.q.weight', 'blocks.21.self_attn.v.bias', 'blocks.21.self_attn.v.weight', 'blocks.22.buffer_encoder.0.weight', 'blocks.22.buffer_encoder.1.weight', 'blocks.22.cam_encoder.weight', 'blocks.22.self_attn.k.bias', 'blocks.22.self_attn.k.weight', 'blocks.22.self_attn.norm_k.weight', 'blocks.22.self_attn.norm_q.weight', 'blocks.22.self_attn.o.bias', 'blocks.22.self_attn.o.weight', 'blocks.22.self_attn.q.bias', 'blocks.22.self_attn.q.weight', 'blocks.22.self_attn.v.bias', 'blocks.22.self_attn.v.weight', 'blocks.23.buffer_encoder.0.weight', 'blocks.23.buffer_encoder.1.weight', 'blocks.23.cam_encoder.weight', 'blocks.23.self_attn.k.bias', 'blocks.23.self_attn.k.weight', 'blocks.23.self_attn.norm_k.weight', 'blocks.23.self_attn.norm_q.weight', 'blocks.23.self_attn.o.bias', 'blocks.23.self_attn.o.weight', 'blocks.23.self_attn.q.bias', 'blocks.23.self_attn.q.weight', 'blocks.23.self_attn.v.bias', 'blocks.23.self_attn.v.weight', 'blocks.24.buffer_encoder.0.weight', 'blocks.24.buffer_encoder.1.weight', 'blocks.24.cam_encoder.weight', 'blocks.24.self_attn.k.bias', 'blocks.24.self_attn.k.weight', 'blocks.24.self_attn.norm_k.weight', 'blocks.24.self_attn.norm_q.weight', 'blocks.24.self_attn.o.bias', 'blocks.24.self_attn.o.weight', 'blocks.24.self_attn.q.bias', 'blocks.24.self_attn.q.weight', 'blocks.24.self_attn.v.bias', 'blocks.24.self_attn.v.weight', 'blocks.25.buffer_encoder.0.weight', 'blocks.25.buffer_encoder.1.weight', 'blocks.25.cam_encoder.weight', 'blocks.25.self_attn.k.bias', 'blocks.25.self_attn.k.weight', 'blocks.25.self_attn.norm_k.weight', 'blocks.25.self_attn.norm_q.weight', 'blocks.25.self_attn.o.bias', 'blocks.25.self_attn.o.weight', 'blocks.25.self_attn.q.bias', 'blocks.25.self_attn.q.weight', 'blocks.25.self_attn.v.bias', 'blocks.25.self_attn.v.weight', 'blocks.26.buffer_encoder.0.weight', 'blocks.26.buffer_encoder.1.weight', 'blocks.26.cam_encoder.weight', 'blocks.26.self_attn.k.bias', 'blocks.26.self_attn.k.weight', 'blocks.26.self_attn.norm_k.weight', 'blocks.26.self_attn.norm_q.weight', 'blocks.26.self_attn.o.bias', 'blocks.26.self_attn.o.weight', 'blocks.26.self_attn.q.bias', 'blocks.26.self_attn.q.weight', 'blocks.26.self_attn.v.bias', 'blocks.26.self_attn.v.weight', 'blocks.27.buffer_encoder.0.weight', 'blocks.27.buffer_encoder.1.weight', 'blocks.27.cam_encoder.weight', 'blocks.27.self_attn.k.bias', 'blocks.27.self_attn.k.weight', 'blocks.27.self_attn.norm_k.weight', 'blocks.27.self_attn.norm_q.weight', 'blocks.27.self_attn.o.bias', 'blocks.27.self_attn.o.weight', 'blocks.27.self_attn.q.bias', 'blocks.27.self_attn.q.weight', 'blocks.27.self_attn.v.bias', 'blocks.27.self_attn.v.weight', 'blocks.28.buffer_encoder.0.weight', 'blocks.28.buffer_encoder.1.weight', 'blocks.28.cam_encoder.weight', 'blocks.28.self_attn.k.bias', 'blocks.28.self_attn.k.weight', 'blocks.28.self_attn.norm_k.weight', 'blocks.28.self_attn.norm_q.weight', 'blocks.28.self_attn.o.bias', 'blocks.28.self_attn.o.weight', 'blocks.28.self_attn.q.bias', 'blocks.28.self_attn.q.weight', 'blocks.28.self_attn.v.bias', 'blocks.28.self_attn.v.weight', 'blocks.29.buffer_encoder.0.weight', 'blocks.29.buffer_encoder.1.weight', 'blocks.29.cam_encoder.weight', 'blocks.29.self_attn.k.bias', 'blocks.29.self_attn.k.weight', 'blocks.29.self_attn.norm_k.weight', 'blocks.29.self_attn.norm_q.weight', 'blocks.29.self_attn.o.bias', 'blocks.29.self_attn.o.weight', 'blocks.29.self_attn.q.bias', 'blocks.29.self_attn.q.weight', 'blocks.29.self_attn.v.bias', 'blocks.29.self_attn.v.weight', 'blocks.3.buffer_encoder.0.weight', 'blocks.3.buffer_encoder.1.weight', 'blocks.3.cam_encoder.weight', 'blocks.3.self_attn.k.bias', 'blocks.3.self_attn.k.weight', 'blocks.3.self_attn.norm_k.weight', 'blocks.3.self_attn.norm_q.weight', 'blocks.3.self_attn.o.bias', 'blocks.3.self_attn.o.weight', 'blocks.3.self_attn.q.bias', 'blocks.3.self_attn.q.weight', 'blocks.3.self_attn.v.bias', 'blocks.3.self_attn.v.weight', 'blocks.30.buffer_encoder.0.weight', 'blocks.30.buffer_encoder.1.weight', 'blocks.30.cam_encoder.weight', 'blocks.30.self_attn.k.bias', 'blocks.30.self_attn.k.weight', 'blocks.30.self_attn.norm_k.weight', 'blocks.30.self_attn.norm_q.weight', 'blocks.30.self_attn.o.bias', 'blocks.30.self_attn.o.weight', 'blocks.30.self_attn.q.bias', 'blocks.30.self_attn.q.weight', 'blocks.30.self_attn.v.bias', 'blocks.30.self_attn.v.weight', 'blocks.31.buffer_encoder.0.weight', 'blocks.31.buffer_encoder.1.weight', 'blocks.31.cam_encoder.weight', 'blocks.31.self_attn.k.bias', 'blocks.31.self_attn.k.weight', 'blocks.31.self_attn.norm_k.weight', 'blocks.31.self_attn.norm_q.weight', 'blocks.31.self_attn.o.bias', 'blocks.31.self_attn.o.weight', 'blocks.31.self_attn.q.bias', 'blocks.31.self_attn.q.weight', 'blocks.31.self_attn.v.bias', 'blocks.31.self_attn.v.weight', 'blocks.32.buffer_encoder.0.weight', 'blocks.32.buffer_encoder.1.weight', 'blocks.32.cam_encoder.weight', 'blocks.32.self_attn.k.bias', 'blocks.32.self_attn.k.weight', 'blocks.32.self_attn.norm_k.weight', 'blocks.32.self_attn.norm_q.weight', 'blocks.32.self_attn.o.bias', 'blocks.32.self_attn.o.weight', 'blocks.32.self_attn.q.bias', 'blocks.32.self_attn.q.weight', 'blocks.32.self_attn.v.bias', 'blocks.32.self_attn.v.weight', 'blocks.33.buffer_encoder.0.weight', 'blocks.33.buffer_encoder.1.weight', 'blocks.33.cam_encoder.weight', 'blocks.33.self_attn.k.bias', 'blocks.33.self_attn.k.weight', 'blocks.33.self_attn.norm_k.weight', 'blocks.33.self_attn.norm_q.weight', 'blocks.33.self_attn.o.bias', 'blocks.33.self_attn.o.weight', 'blocks.33.self_attn.q.bias', 'blocks.33.self_attn.q.weight', 'blocks.33.self_attn.v.bias', 'blocks.33.self_attn.v.weight', 'blocks.34.buffer_encoder.0.weight', 'blocks.34.buffer_encoder.1.weight', 'blocks.34.cam_encoder.weight', 'blocks.34.self_attn.k.bias', 'blocks.34.self_attn.k.weight', 'blocks.34.self_attn.norm_k.weight', 'blocks.34.self_attn.norm_q.weight', 'blocks.34.self_attn.o.bias', 'blocks.34.self_attn.o.weight', 'blocks.34.self_attn.q.bias', 'blocks.34.self_attn.q.weight', 'blocks.34.self_attn.v.bias', 'blocks.34.self_attn.v.weight', 'blocks.35.buffer_encoder.0.weight', 'blocks.35.buffer_encoder.1.weight', 'blocks.35.cam_encoder.weight', 'blocks.35.self_attn.k.bias', 'blocks.35.self_attn.k.weight', 'blocks.35.self_attn.norm_k.weight', 'blocks.35.self_attn.norm_q.weight', 'blocks.35.self_attn.o.bias', 'blocks.35.self_attn.o.weight', 'blocks.35.self_attn.q.bias', 'blocks.35.self_attn.q.weight', 'blocks.35.self_attn.v.bias', 'blocks.35.self_attn.v.weight', 'blocks.36.buffer_encoder.0.weight', 'blocks.36.buffer_encoder.1.weight', 'blocks.36.cam_encoder.weight', 'blocks.36.self_attn.k.bias', 'blocks.36.self_attn.k.weight', 'blocks.36.self_attn.norm_k.weight', 'blocks.36.self_attn.norm_q.weight', 'blocks.36.self_attn.o.bias', 'blocks.36.self_attn.o.weight', 'blocks.36.self_attn.q.bias', 'blocks.36.self_attn.q.weight', 'blocks.36.self_attn.v.bias', 'blocks.36.self_attn.v.weight', 'blocks.37.buffer_encoder.0.weight', 'blocks.37.buffer_encoder.1.weight', 'blocks.37.cam_encoder.weight', 'blocks.37.self_attn.k.bias', 'blocks.37.self_attn.k.weight', 'blocks.37.self_attn.norm_k.weight', 'blocks.37.self_attn.norm_q.weight', 'blocks.37.self_attn.o.bias', 'blocks.37.self_attn.o.weight', 'blocks.37.self_attn.q.bias', 'blocks.37.self_attn.q.weight', 'blocks.37.self_attn.v.bias', 'blocks.37.self_attn.v.weight', 'blocks.38.buffer_encoder.0.weight', 'blocks.38.buffer_encoder.1.weight', 'blocks.38.cam_encoder.weight', 'blocks.38.self_attn.k.bias', 'blocks.38.self_attn.k.weight', 'blocks.38.self_attn.norm_k.weight', 'blocks.38.self_attn.norm_q.weight', 'blocks.38.self_attn.o.bias', 'blocks.38.self_attn.o.weight', 'blocks.38.self_attn.q.bias', 'blocks.38.self_attn.q.weight', 'blocks.38.self_attn.v.bias', 'blocks.38.self_attn.v.weight', 'blocks.39.buffer_encoder.0.weight', 'blocks.39.buffer_encoder.1.weight', 'blocks.39.cam_encoder.weight', 'blocks.39.self_attn.k.bias', 'blocks.39.self_attn.k.weight', 'blocks.39.self_attn.norm_k.weight', 'blocks.39.self_attn.norm_q.weight', 'blocks.39.self_attn.o.bias', 'blocks.39.self_attn.o.weight', 'blocks.39.self_attn.q.bias', 'blocks.39.self_attn.q.weight', 'blocks.39.self_attn.v.bias', 'blocks.39.self_attn.v.weight', 'blocks.4.buffer_encoder.0.weight', 'blocks.4.buffer_encoder.1.weight', 'blocks.4.cam_encoder.weight', 'blocks.4.self_attn.k.bias', 'blocks.4.self_attn.k.weight', 'blocks.4.self_attn.norm_k.weight', 'blocks.4.self_attn.norm_q.weight', 'blocks.4.self_attn.o.bias', 'blocks.4.self_attn.o.weight', 'blocks.4.self_attn.q.bias', 'blocks.4.self_attn.q.weight', 'blocks.4.self_attn.v.bias', 'blocks.4.self_attn.v.weight', 'blocks.5.buffer_encoder.0.weight', 'blocks.5.buffer_encoder.1.weight', 'blocks.5.cam_encoder.weight', 'blocks.5.self_attn.k.bias', 'blocks.5.self_attn.k.weight', 'blocks.5.self_attn.norm_k.weight', 'blocks.5.self_attn.norm_q.weight', 'blocks.5.self_attn.o.bias', 'blocks.5.self_attn.o.weight', 'blocks.5.self_attn.q.bias', 'blocks.5.self_attn.q.weight', 'blocks.5.self_attn.v.bias', 'blocks.5.self_attn.v.weight', 'blocks.6.buffer_encoder.0.weight', 'blocks.6.buffer_encoder.1.weight', 'blocks.6.cam_encoder.weight', 'blocks.6.self_attn.k.bias', 'blocks.6.self_attn.k.weight', 'blocks.6.self_attn.norm_k.weight', 'blocks.6.self_attn.norm_q.weight', 'blocks.6.self_attn.o.bias', 'blocks.6.self_attn.o.weight', 'blocks.6.self_attn.q.bias', 'blocks.6.self_attn.q.weight', 'blocks.6.self_attn.v.bias', 'blocks.6.self_attn.v.weight', 'blocks.7.buffer_encoder.0.weight', 'blocks.7.buffer_encoder.1.weight', 'blocks.7.cam_encoder.weight', 'blocks.7.self_attn.k.bias', 'blocks.7.self_attn.k.weight', 'blocks.7.self_attn.norm_k.weight', 'blocks.7.self_attn.norm_q.weight', 'blocks.7.self_attn.o.bias', 'blocks.7.self_attn.o.weight', 'blocks.7.self_attn.q.bias', 'blocks.7.self_attn.q.weight', 'blocks.7.self_attn.v.bias', 'blocks.7.self_attn.v.weight', 'blocks.8.buffer_encoder.0.weight', 'blocks.8.buffer_encoder.1.weight', 'blocks.8.cam_encoder.weight', 'blocks.8.self_attn.k.bias', 'blocks.8.self_attn.k.weight', 'blocks.8.self_attn.norm_k.weight', 'blocks.8.self_attn.norm_q.weight', 'blocks.8.self_attn.o.bias', 'blocks.8.self_attn.o.weight', 'blocks.8.self_attn.q.bias', 'blocks.8.self_attn.q.weight', 'blocks.8.self_attn.v.bias', 'blocks.8.self_attn.v.weight', 'blocks.9.buffer_encoder.0.weight', 'blocks.9.buffer_encoder.1.weight', 'blocks.9.cam_encoder.weight', 'blocks.9.self_attn.k.bias', 'blocks.9.self_attn.k.weight', 'blocks.9.self_attn.norm_k.weight', 'blocks.9.self_attn.norm_q.weight', 'blocks.9.self_attn.o.bias', 'blocks.9.self_attn.o.weight', 'blocks.9.self_attn.q.bias', 'blocks.9.self_attn.q.weight', 'blocks.9.self_attn.v.bias', 'blocks.9.self_attn.v.weight', 'clean_patch_embeddings.0.bias', 'clean_patch_embeddings.0.weight', 'clean_patch_embeddings.1.bias', 'clean_patch_embeddings.1.weight', 'clean_patch_embeddings.2.bias', 'clean_patch_embeddings.2.weight', 'clean_patch_embeddings.3.bias', 'clean_patch_embeddings.3.weight', 'clean_patch_embeddings.4.bias', 'clean_patch_embeddings.4.weight', 'clean_patch_embeddings.5.bias', 'clean_patch_embeddings.5.weight', 'patch_embedding.bias', 'patch_embedding.weight']
[05-01 07:35:14|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on Creating PyTorch model: 1.0963 seconds
[05-01 07:35:14|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on Creating PyTorch model and ema if enabled: 5.0339 seconds
[05-01 07:35:14|INFO|lyra_2/_ext/imaginaire/utils/misc.py:277:wrapper] Time spent on DiffusionModel: set_up_model: 5.0341 seconds
[05-01 07:35:14|INFO|lyra_2/_src/models/lyra2_model.py:441:_init_lyra2_metadata] Lyra2: splits=[1, 4, 1, 16, 2, 1], kernel_sizes=[1, 2, 1, 4, 2, 1], kernel_types=['k', 's', 's', 'k', 'k', 'k'], new_latent_frames=20
[05-01 07:35:14|INFO|lyra_2/_src/models/lyra2_model.py:118:__init__] Lyra2Model spatial history positions: (1, 2, 3, 4, 5), spatial_memory_stride=8, spatial_memory_skip_recent=16
[05-01 07:35:14|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on instantiate model: 5.7004 seconds
[05-01 07:35:14|INFO|lyra_2/_src/utils/model_loader.py:79:load_model_from_checkpoint] Loading model from dcp checkpoint checkpoints/model
/home/******/anaconda3/envs/lyra2/lib/python3.10/site-packages/torch/distributed/checkpoint/state_dict_loader.py:153: UserWarning: torch.distributed is disabled, unavailable or uninitialized, assuming the intent is to load in a single process.
  warnings.warn(
[05-01 07:35:48|WARNING|lyra_2/_src/models/wan_t2v_model.py:988:load_state_dict] load model in non-strict mode
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.0.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.0.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.0.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.1.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.1.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.1.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.2.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.2.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.2.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.3.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.3.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.3.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.4.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.4.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.4.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.5.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.5.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.5.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.6.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.6.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.6.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.7.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.7.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.7.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.8.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.8.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.8.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.9.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.9.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.9.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.10.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.10.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.10.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.11.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.11.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.11.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.12.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.12.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.12.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.13.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.13.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.13.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.14.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.14.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.14.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.15.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.15.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.15.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.16.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.16.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.16.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.17.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.17.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.17.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.18.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.18.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.18.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.19.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.19.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.19.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.20.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.20.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.20.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.21.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.21.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.21.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.22.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.22.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.22.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.23.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.23.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.23.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.24.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.24.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.24.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.25.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.25.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.25.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.26.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.26.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.26.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.27.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.27.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.27.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.28.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.28.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.28.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.29.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.29.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.29.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.30.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.30.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.30.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.31.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.31.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.31.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.32.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.32.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.32.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.33.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.33.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.33.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.34.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.34.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.34.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.35.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.35.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.35.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.36.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.36.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.36.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.37.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.37.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.37.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.38.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.38.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.38.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.39.self_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.39.cross_attn.attn_op._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_ext/imaginaire/utils/checkpointer.py:321:non_strict_load_model] Skipping key blocks.39.cross_attn.attn_op_image._extra_state introduced by TransformerEngine for FP8 in the checkpoint.
[05-01 07:35:48|WARNING|lyra_2/_src/models/wan_t2v_model.py:989:load_state_dict] [RANK 0]_IncompatibleKeys(missing_keys=[], unexpected_keys=[], incorrect_shapes=[])
[05-01 07:35:49|INFO|lyra_2/_src/models/wan_t2v_model.py:408:load_lora_weights] Injecting LoRA as from checkpoints/lora/realism_boost.safetensors, rank: 32, lora_alpha: 32, lora_bias: False, target_modules: {'ffn.2', 'q', 'ffn.0', 'k', 'o', 'v'}
[05-01 07:35:51|INFO|lyra_2/_src/models/wan_t2v_model.py:479:add_lora_to_model] Trainable parameters after adding LoRA: 0 / Total parameters: 0
[05-01 07:35:51|INFO|lyra_2/_src/models/wan_t2v_model.py:420:load_lora_weights] Injected LoRA weights as from checkpoints/lora/realism_boost.safetensors
[05-01 07:35:51|INFO|lyra_2/_src/models/wan_t2v_model.py:533:load_weights_to_lora] 676 parameters are loaded from None. 0 parameters are unexpected.
[05-01 07:35:51|INFO|lyra_2/_src/models/wan_t2v_model.py:429:load_lora_weights] Loaded LoRA weights from checkpoints/lora/realism_boost.safetensors
[05-01 07:35:51|WARNING|lyra_2/_src/models/wan_t2v_model.py:375:load_lora_weights] LoRA ['realism_boost--safetensors'] loaded, loading detail_enhancer--safetensors will override it!
[05-01 07:35:51|INFO|lyra_2/_src/models/wan_t2v_model.py:408:load_lora_weights] Injecting LoRA as from checkpoints/lora/detail_enhancer.safetensors, rank: 32, lora_alpha: 32, lora_bias: False, target_modules: {'ffn.2', 'q', 'ffn.0', 'k', 'o', 'v'}
/home/******/anaconda3/envs/lyra2/lib/python3.10/site-packages/peft/tuners/tuners_utils.py:196: UserWarning: Already found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!
  warnings.warn(
[05-01 07:35:52|INFO|lyra_2/_src/models/wan_t2v_model.py:479:add_lora_to_model] Trainable parameters after adding LoRA: 0 / Total parameters: 0
[05-01 07:35:52|INFO|lyra_2/_src/models/wan_t2v_model.py:420:load_lora_weights] Injected LoRA weights as from checkpoints/lora/detail_enhancer.safetensors
[05-01 07:35:53|INFO|lyra_2/_src/models/wan_t2v_model.py:533:load_weights_to_lora] 676 parameters are loaded from None. 0 parameters are unexpected.
[05-01 07:35:53|INFO|lyra_2/_src/models/wan_t2v_model.py:429:load_lora_weights] Loaded LoRA weights from checkpoints/lora/detail_enhancer.safetensors
[05-01 07:35:53|INFO|lyra_2/_src/models/wan_t2v_model.py:571:set_weights_and_activate_adapters] Set weights: [0.4, 0.4], and activate adapters: ['realism_boost--safetensors', 'detail_enhancer--safetensors']
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1101:enable_selective_checkpoint] Enable selective checkpoint with block_wise, for every 1 blocks. Total blocks: 40
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 0
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 1
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 2
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 3
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 4
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 5
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 6
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 7
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 8
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 9
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 10
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 11
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 12
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 13
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 14
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 15
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 16
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 17
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 18
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 19
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 20
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 21
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 22
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 23
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 24
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 25
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 26
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 27
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 28
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 29
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 30
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 31
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 32
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 33
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 34
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 35
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 36
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 37
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 38
[05-01 07:35:53|INFO|lyra_2/_src/networks/wan2pt1_lyra2.py:1107:enable_selective_checkpoint] Enable selective checkpoint for block 39
[05-01 07:35:53|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:538:<module>] Casted model.net to dtype=torch.bfloat16
[05-01 07:35:53|INFO|lyra_2/_src/inference/depth_utils.py:204:load_da3_model] Loading DA3 model with custom checkpoint: checkpoints/recon/model.pt
[05-01 07:35:53|INFO|lyra_2/_src/inference/depth_utils.py:158:load_da3_from_custom_checkpoint] Initializing DA3 architecture from local config: da3nested-giant-large (requested=depth-anything/DA3NESTED-GIANT-LARGE-1.1)
[INFO ] using SwiGLU layer as FFN
[INFO ] using MLP layer as FFN
[05-01 07:36:00|INFO|lyra_2/_src/inference/depth_utils.py:164:load_da3_from_custom_checkpoint] Loading custom checkpoint from checkpoints/recon/model.pt
[05-01 07:36:07|INFO|lyra_2/_src/inference/depth_utils.py:190:load_da3_from_custom_checkpoint] Loaded 1383 parameters from custom checkpoint
/home/******/anaconda3/envs/lyra2/lib/python3.10/site-packages/moge/model/v1.py:172: UserWarning: The following deprecated/invalid arguments are ignored: {'output_mask': True, 'split_head': True}
  warnings.warn(f"The following deprecated/invalid arguments are ignored: {deprecated_kwargs}")
[05-01 07:36:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:572:<module>] MoGe model loaded for depth scale alignment.
[05-01 07:36:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:598:<module>] Processing [0]: assets/samples/04.png
[05-01 07:36:11|INFO|lyra_2/_ext/imaginaire/utils/misc.py:150:set_random_seed] Using random seed 1.
[05-01 07:36:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:610:<module>] Running DA3 single-image depth...
[INFO ] Processed Images Done taking 0.01938629150390625 seconds. Shape:  torch.Size([1, 3, 476, 826])
[INFO ] Model Forward Pass Done. Time: 0.3517425060272217 seconds
[INFO ] Conversion to Prediction Done. Time: 0.0008704662322998047 seconds
[05-01 07:36:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:620:<module>] Aligning DA3 depth to MoGe scale...
[05-01 07:36:12|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:650:<module>] Global inverse-depth scale factor: 5.884039402008057
[05-01 07:36:12|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:679:<module>] Loaded caption from assets/samples/04.txt: A cinematic drone shot glides forward through the center of a sun-drenched, ancient stone amphitheater. The weathered limestone blocks, crumbling arches, and dense green ivy are frozen in time. Every leaf, shadow, and dust particle is perfectly still. The warm, golden sunlight remains constant across the textured ruins and cobblestone path. As the camera advances, more of the circular arena and tiered seating are revealed, maintaining the same overgrown, historical aesthetic. All environmental elements, including the distant foliage and sky, are entirely motionless, creating a silent, statuesque world where only the camera’s perspective shifts.
[05-01 07:36:12|INFO|lyra_2/_src/inference/get_t5_emb.py:509:__init__] loading ./checkpoints/text_encoder/encoder.pth
/home/******/lyra/Lyra-2/lyra_2/_ext/imaginaire/utils/easy_io/easy_io.py:976: UserWarning: fast_backend is not supported by the backend, type <class 'lyra_2._ext.imaginaire.utils.easy_io.backends.local_backend.LocalBackend'> fallback to normal get
  warnings.warn(
[05-01 07:36:22|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:716:<module>] === Generating ZOOM-IN video (horizontal_zoom right str=0.5, N=81) ===
[05-01 07:36:29|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:1271:run_lyra2_sample] 04_zoom_in
  0%|                                                                    | 0/1 [00:00<?, ?it/s][05-01 07:36:32|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - corruptor: 2.5514 seconds
[05-01 07:36:34|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - encode camera tail: 2.3510 seconds
[05-01 07:36:34|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - build: 4.9030 seconds
[05-01 07:36:34|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on plucker_condition - build: 0.0013 seconds
[05-01 07:36:34|INFO|lyra_2/_src/models/lyra2_model.py:2244:_prepare_lyra2_inputs] Buffer hist padding: ['pos0:f1k=zeros', 'pos1:f4s=spatial_coord', 'pos2:f1s=spatial_coord', 'pos3:f16k=zeros', 'pos4:f2k=zeros', 'pos5:f1k=zeros'], C_buf=1280, _max_spatial=5
100%|██████████████████████████████████████████████████████████| 1/1 [20:45<00:00, 1245.89s/it]
[05-01 07:57:16|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:741:<module>] === Generating ZOOM-OUT video (horizontal_zoom left str=1.5, N=241) ===
[05-01 07:57:16|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:401:_generate_one_direction] 04_zoom_out [upward_tilt] Added upward ratio=0.150, max_upward=0.1444
[05-01 07:57:27|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:1271:run_lyra2_sample] 04_zoom_out
  0%|                                                                    | 0/3 [00:00<?, ?it/s][05-01 07:57:30|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - corruptor: 2.5127 seconds
[05-01 07:57:33|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - encode camera tail: 3.3461 seconds
[05-01 07:57:33|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - build: 5.8592 seconds
[05-01 07:57:33|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on plucker_condition - build: 0.0047 seconds
[05-01 07:57:34|INFO|lyra_2/_src/models/lyra2_model.py:2244:_prepare_lyra2_inputs] Buffer hist padding: ['pos0:f1k=zeros', 'pos1:f4s=spatial_coord', 'pos2:f1s=spatial_coord', 'pos3:f16k=zeros', 'pos4:f2k=zeros', 'pos5:f1k=zeros'], C_buf=1280, _max_spatial=5
[INFO ] Processed Images Done taking 0.08415818214416504 seconds. Shape:  torch.Size([10, 3, 476, 826])
median_dist: 0.12976691126823425
[INFO ] Using camera conditions provided by the user
[INFO ] Model Forward Pass Done. Time: 1.358698844909668 seconds
[INFO ] Conversion to Prediction Done. Time: 0.011897802352905273 seconds
 33%|███████████████████▎                                      | 1/3 [20:47<41:35, 1247.79s/it][05-01 08:18:15|INFO|lyra_2/_src/models/lyra2_model.py:2788:retrieve] Sparse3DCache.retrieve(max_coverage): pre-covering pixels from temporally closest frame_id=80 (cand_idx=10, pixels=3984)
[05-01 08:18:24|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - corruptor: 8.8253 seconds
[05-01 08:18:27|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - encode camera tail: 2.9445 seconds
[05-01 08:18:27|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - build: 11.7775 seconds
[05-01 08:18:27|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on plucker_condition - build: 0.0012 seconds
[05-01 08:18:27|INFO|lyra_2/_src/models/lyra2_model.py:2244:_prepare_lyra2_inputs] Buffer hist padding: ['pos0:f1k=zeros', 'pos1:f4s=spatial_coord', 'pos2:f1s=spatial_coord', 'pos3:f16k=zeros', 'pos4:f2k=zeros', 'pos5:f1k=zeros'], C_buf=1280, _max_spatial=5
[INFO ] Processed Images Done taking 0.13765311241149902 seconds. Shape:  torch.Size([10, 3, 476, 826])
median_dist: 0.12976691126823425
[INFO ] Using camera conditions provided by the user
[INFO ] Model Forward Pass Done. Time: 1.281848669052124 seconds
[INFO ] Conversion to Prediction Done. Time: 0.009133577346801758 seconds
 67%|██████████████████████████████████████▋                   | 2/3 [41:34<20:47, 1247.45s/it][05-01 08:39:02|INFO|lyra_2/_src/models/lyra2_model.py:2788:retrieve] Sparse3DCache.retrieve(max_coverage): pre-covering pixels from temporally closest frame_id=160 (cand_idx=20, pixels=6025)
[05-01 08:39:11|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - corruptor: 8.1967 seconds
[05-01 08:39:13|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - encode camera tail: 2.8566 seconds
[05-01 08:39:13|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on camera_pose_condition - build: 11.0614 seconds
[05-01 08:39:13|INFO|lyra_2/_ext/imaginaire/utils/misc.py:266:__exit__] Time spent on plucker_condition - build: 0.0039 seconds
[05-01 08:39:14|INFO|lyra_2/_src/models/lyra2_model.py:2244:_prepare_lyra2_inputs] Buffer hist padding: ['pos0:f1k=zeros', 'pos1:f4s=spatial_coord', 'pos2:f1s=spatial_coord', 'pos3:f16k=zeros', 'pos4:f2k=zeros', 'pos5:f1k=zeros'], C_buf=1280, _max_spatial=5
100%|████████████████████████████████████████████████████████| 3/3 [1:02:35<00:00, 1251.99s/it]
[05-01 09:00:04|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:343:save_output] video_array.shape: (81, 960, 832, 3) value: 255, 0, save to outputs/zoomgs/04/zoom_in.mp4
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[05-01 09:00:06|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:361:save_output] save video to outputs/zoomgs/04/zoom_in.mp4
[05-01 09:00:06|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:777:<module>] Saved zoom_in video: outputs/zoomgs/04/zoom_in.mp4
[05-01 09:00:07|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:343:save_output] video_array.shape: (241, 960, 832, 3) value: 255, 0, save to outputs/zoomgs/04/zoom_out.mp4
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[05-01 09:00:11|INFO|lyra_2/_src/inference/lyra2_ar_inference.py:361:save_output] save video to outputs/zoomgs/04/zoom_out.mp4
[05-01 09:00:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:777:<module>] Saved zoom_out video: outputs/zoomgs/04/zoom_out.mp4
[05-01 09:00:11|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:787:<module>] Combined video: 322 frames from both directions
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[05-01 09:00:13|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:791:<module>] Saved combined video: outputs/zoomgs/videos/04.mp4
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[05-01 09:00:16|INFO|lyra_2/_src/inference/lyra2_zoomgs_inference.py:811:<module>] Done.

real    85m18.256s
user    112m12.362s
sys     44m42.856s