Cristian Canton

I am the head of Responsible AI (RAI) at Meta, where I support multiple organizations related to AI robustness, fairness, transparency, and legitimacy. Recently, have expanded my scope to cover all aspects of RAI for Generative AI.

Erdős number: 3.

LinkedIn /  Threads /  Twitter /  Google Scholar /  Email

I'm interested in computer vision, machine learning, security in AI, adversarial attacks and fairness.

TL;DR. Meet GOAT (Generative Offensive Agent Tester), an automatic red teaming system to simulate adversarial conversations and identify vulnerabilities in large language models (LLMs). It uses a range of adversarial prompting techniques to test the limits of LLMs, automating the red teaming process to make it more efficient and effective.

TL;DR. A collection of open source multi-modal multi-lingual GenAI models (7B, 70B, 405B) with SOTA capabilities. Among my contributions, to make it safe and robust via red teaming.

TL;DR. A summary paper evaluating fairness, privacy and regulatory aspects of over 100 datasets. A work that sets the standards on defining what a responsible dataset is.

TL;DR. This paper proposes a meta-learning framework for group-level fairness in machine learning, which incorporates a multi-player cooperative bargaining game to resolve hypergradient conflicts that may arise when fairness objectives are integrated into the model training process.

TL;DR. This paper presents the first study on human based red teaming for Machine Translation,marking a significant step towards understanding and improving the performance of translation models.

TL;DR. In this paper we prove that the pedestrian tracking task can be done in a privacy preserving manner by replacing individuals by time-consistent full-body person synthetic.

TL;DR. Code Llama is an AI model built on top of Llama 2, fine-tuned for generating and discussing code. First of it kind!

TL;DR. This paper introduces Llama2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.

TL;DR. The first framework that generalizes visual prompting with test-time adaptation. VPA introduces a small number of learnable tokens, enabling fully test-time and storage-efficient adaptation without necessitating source-domain information.

TL;DR. A diverse, large benchmark for measuring fairness and robustness in audio/vision/speech models. A bold step forward from our Casual Conversations v1 dataset.

TL;DR. Unintended decision rules that are unable to generalize, shortcuts, are very prevalent in ML models. This paper how mitigations for shortcuts are a whack-a-model game, for now.

TL;DR. A first paper to design a ranking function that gives useful and personalized post recommendations in social media.

TL;DR. Leverage diffusion process based generative models to synthesize novel images from low-density regions.

TL;DR. A pioneering work on hidding images on audio, showing real results retrieving images from recorded audio waves.

TL;DR. A quantitative and qualitative analysis of the top submissions to the 2021 Image Similarity Challenge.

TL;DR. Prove how good are you at finding modified copies of images. $200K in prizes!

TL;DR. Multimodal understanding (image+text) can be attacked in a gray-box fashing. Examples on hate memes included!

TL;DR. Undertand what areas of an image are more vulnerable to increase uncertainty of models and eventually making them to produce wrong decisions.

TL;DR. Hashing algorithms (in general) can be easily compromised by introducing collisions.

TL;DR. A large dataset (10TB, 30+ days of video, 3K people, 15 mins per person) featuring a diversity of actors with age, gender and skin tone annotations.

TL;DR. Defusing deepfake detection by crafting universal adversarial perturbations, transferable across different detection models.

TL;DR. Creation of adversarial additive patterns against open sourced deepfake detection systems turns out to be very effective.

TL;DR. Create embeddings that are privacy preserving thus non-reversible to obtain the original image.

TL;DR. A comprehensive survey of AI techniques applied in social networks to preserve integrity and preserve users from bad experiences.

TL;DR. Up to today, the largest deepfake dataset. A $1M competition!

TL;DR. A small preview version of the DFDC dataset to come later.

TL;DR. Multimodal understanding of memes can allow better detection of image+text hate speech.

TL;DR. Leverage ego-centric camera footage to understand long time span scene interactions, specially for social style signals..

TL;DR. Given an example of how your eyes look, this algorithm can take a pic with your eyes closed and open them. GAN-powered.

TL;DR. Use an ego-camera, process multimodal inputs to do long time span scene understanding, specially for social style signals.

TL;DR. An unsupervised framework to extract semantically rich features for video representation inspired by how the human visual system groups objects based on motion cues.

TL;DR. Estimate of saliency using a GAN, probably the first article to do it!

TL;DR. Some strategies to combine multiple subjective labels for human emotions in a way that CNNs can be trained effectively.

TL;DR. Ensambles of CNN embeddings allow for high accuracy emotion recognition on videos.

TL;DR. While tracking multiple targets, comparing detections using a Siamese network improves overall performance.

TL;DR. A survey on marker based motion capture technologies for pose estimation.

TL;DR. Tracking multiple object on the 3D domain by voxel reconstruction and particle filtering techniques.

TL;DR. A new Monte Carlo method applied to recover articulated structured (human body) exploiting hierarchies of complexity.

TL;DR. Combination of motion cues, position and audio allows detection of events within a SmartRoom environment.

TL;DR. Survey on image and video processing techniques for human computer interaction in SmartRooms.

TL;DR. Generating virtual orthogonal views derived from 3D voxels allow better action recognition on 2D projections.

TL;DR. Marker based motion tracking using annealed particle filtering.

TL;DR. Understanding actions directly on 3D+time hypervolumes using the Radon transform.

TL;DR. Conditioning particle filtering to run on surface of volumes ensure better tracking through a new occupancy resampling strategy.

TL;DR. An approach to a fair estimation of human pose estimation through the Kolmogorov-Smirnov test.

TL;DR. A survey of the technologies for head pose estimation in multimodal environments.

TL;DR. A survey on activity recognition technologies applied to SmartRooms.

TL;DR. Marker based motion tracking using annealed particle filtering.

TL;DR. Acoustic event detection using multiple cues and specific features using HMM.

TL;DR. Combination of motion cues, position and audio allows detection of events within a SmartRoom environment.

TL;DR. Markerless human body capture leveraging annealed particle filters prove to be effective and robust to occlusions.

TL;DR. Adding color to 3D voxel representations improve multi-person tracking.

TL;DR. Leveraging spatial information of people in a SmartRoom improves acoustic event detection.

TL;DR. Multimodal data fusion towards acoustic events understanding in SmartRooms.

TL;DR. New Montecarlo method applied to recover articulated structured (human body) exploiting hierarchies of complexity.

TL;DR. Estimate the spatial focus of attention of a group of people leveraging their head orientation.

TL;DR. Analysis and synthesis of motion patterns correlating audio and multiview information.

TL;DR. Analyze audio to make an avatar dance to the music.

TL;DR. Human dance patterns generation directly from music.

TL;DR. A system to combine multiple vision signals to detect unusual/outlier behaviors.

TL;DR. Define head templates and use them in a particle filtering context to estimate head position.

TL;DR. Comparison of two tracking aproaches (heuristic vs particle filtering) on the CLEAR dataset.

TL;DR. Person tracking using particle filters directly on voxels, handling multiple objects and delivering high accuracy tracking.

TL;DR. Monte Carlo techniques applied to audiovisual data fusion for head estimation and attention estimation purposes.

TL;DR. Head pose estimation using both multicamera and microphone arrays signals increase robustness and accuracy of estimation.

TL;DR. Analyze 3D volumes using a human body model to recognize actions in multicamera scenarios.

TL;DR. Use multiple cameras to create voxels and recognize actions directly from the 3D.

TL;DR. Head position estimation using multiview face appearance and its evaluation against the CLEAR dataset.

TL;DR. Multiple modalities always improve tracking, specially of humans within a SmartRoom.

TL;DR. Head orientation estimation by 3D backpropagation of detected skin patches on multiple views.

TL;DR. Kalman filtering of 3D points directly from 2D projections; robust to occlusions by analyzing outliers. Simple yet effective!

TL;DR. Multimodal understanding in SmartRooms will enable HCI scenarios to make meetings more effective.

TL;DR. Tracking multiple 3D points in a multiview scenario using a greedy Bayesian approach. Useful for low compute scenarios and SmartRooms.

TL;DR. Conductance functions used in anisotropic diffusion schemes may have relevant impact on the final result.

TL;DR. Use overcomplete representations to represent fingerprints and match them against others.

TL;DR. An old shcool image filtering technique based on the maximization of the similarities between pixels in the filtering window.

TL;DR. When running anisotropic diffusion filters on images, not all conduction functions were born equal.

TL;DR. Human motion capture using hierarchical models and Monte Carlo optimization techniques. No fancy ML involved, just math and pixels.

TL;DR. Overcomplete representations allow high compression of images/videos. Exploiting texture and edge priors into the process yield to dramatic improvements.