Projects Mar 2025 Solo

Visionary GenAI

Three computer vision tasks using modern GenAI models: biometric spoof detection (95.5% accuracy), CLIP-based semantic image retrieval, and Stable Diffusion generation.

  • Python
  • PyTorch
  • Transformers
  • CLIP
  • Stable Diffusion
  • Hugging Face

Three self-contained experiments applying Vision Transformers, CLIP, and Stable Diffusion to distinct problem classes.

Part 1: Biometric spoof detection

Fine-tuned google/vit-base-patch16-224-in21k as a binary live/spoof classifier on the CelebA-Spoof dataset. The target scenario: an attacker holds a printed or digital photo in front of a facial authentication camera.

95.5%

Test accuracy

~67K

Images

80/10/10 split

5

Epochs

Setup
Batch 64; lr 5e-5; AdamW; CrossEntropyLoss; patch size 16; input 224x224

ViT’s patch-based self-attention captures spatial relationships across facial regions that CNNs at this scale struggle with. The pretrained weights transfer well: five epochs of fine-tuning on 53K samples was enough to reach 95.5% on the held-out test set. Custom test images (real photo vs photo of a phone screen showing the same face) both classified correctly.

Built a zero-shot text-to-image retrieval engine over the COCO val set (~5K images with captions). A text query is encoded by CLIP’s text encoder; each image in the corpus is encoded by CLIP’s image encoder. Dot product similarity between the two embeddings ranks results.

No training required: CLIP’s joint text-image pre-training means the embedding space already aligns language and vision semantics.

Part 3: Stable Diffusion experiments

Explored image-to-image and text-to-image pipelines using dreamlike-art/dreamlike-photoreal-2.0 and stabilityai/stable-diffusion-2-1. Systematically varied strength, guidance_scale, and num_inference_steps to map their effects on output fidelity and adherence to the source.

Low strength (0.3)
Preserves source structure at the cost of style depth; the model barely departs from the original.
Low guidance_scale (2)
Weakens prompt adherence and produces loosely related outputs.
High strength + guidance_scale (20)
Yields the cleanest style transfers but discards source-specific details like color and texture. Requires 40+ inference steps.