source: kdnuggets: local video summarization pipeline: processing frames with smolvlm2-2.2b
level: technical
smolvlm2-2.2b-instruct is a vision-language model that runs on 5.2 gb of gpu ram, fitting on an rtx 3060 or macbook pro m2. it compresses each image patch to 81 tokens using a pixel shuffle strategy, allowing 50 frames to use about 4,050 image tokens in one inference call. this token efficiency makes it 3.3 to 4.5 times faster at prefill and 7.5 to 16 times faster at generation than qwen2-vl-2b. on the video-mme benchmark, it scores 52.1, outperforming other 2b-scale models.
the pipeline extracts frames from any video file using uniform or keyframe sampling. uniform sampling spreads frames evenly across the video, good for meetings and lectures. keyframe sampling captures frames only when visual content changes significantly, useful for surveillance or highlights. the frame extractor outputs pil images with timestamps, capped at 50 frames to stay within vram limits. the model then describes each frame or batch of frames, producing scene descriptions, key moments, and action items.
the full project builds a meeting recording summarizer that outputs a structured json summary. it includes per-frame descriptions, key moments with timestamps, action items, and a final narrative. the same code works for lectures and surveillance footage without changes. the model loads with flash attention 2 on cuda for faster multi-image processing. the pipeline is designed for developers who want to process hours of video on their own workstation without cloud costs.
why it matters: it lets data scientists and developers summarize long videos locally on consumer hardware, avoiding cloud api costs and privacy concerns.
source: kdnuggets: local video summarization pipeline: processing frames with smolvlm2-2.2b