My solution was made early on in the competition and uses only video data. It held the first place initially, though got surpassed later on. In hindsight I think I could have benefitted from using the keypoints as well.
I used an ensemble of pre-trained vision models by concatenating the output features of the vision models (resnet18 and MobileNetV3-Small). This results in a large vector which is then reduced to size 64 by PCA for the beetles challenge, which is the final embedding. For the mouse part of the challenge I instead reduce the size down to 32 in a similar way, and then concatenate to this the difference of the feature vector from 40 frames in the past and 40 frames in the future (window size of 80 as in the solution of @edhayes1) to also have dynamic information present in the embedding.