Problem statement

Animal behavior research increasingly relies on large audio, video, and biologger datasets, but manual coding is slow, subjective, and fragmented across formats – limiting cross-species analyses and large-scale synthesis. Foundation models make AI-assisted coding feasible with modest labeled data, yet generalization across species and unified workflows remain unmet needs. We will harmonize existing, human-coded datasets from multiple BeCogBio labs and fine-tune open, pre-trained models to deliver an open-source pipeline and app for scalable, cross-species behavioral coding.

Research plan

We will build a shared, RDM-compliant data hub (BeCogBioBank), test and fine-tune open foundation models, and validate cross-species transfer on two core avian testbeds. The project delivers an open framework and web app (AIHAB), code/models on GitHub, and early publications:

Data harmonization

• Combine and standardize multi-lab video, audio, and sensor data into a shared repository
• Prepare datasets for efficient training and validation

Model development and evaluation

• Adapt and compare suitable open AI approaches for behavior classification
• Validate results against held-out human-coded ground truth

Core testbeds

• Raven to crow (video ethograms)
• Golden eagle to crowned eagle (video + accelerometer)

Deliverables and dissemination

• Project workshop in Vienna (October) and final dissemination event
• Methods paper on ground-truthing
• AIHAB: open framework and web app; code and models on GitHub