In this blog post, I cover one of the awarded papers in NeurIPS 2022. This paper presents LAION-5B, a dataset consisting of 5.9 billion image-text pairs, to further push the scale of open datasets for training and studying state-of-the-art language-vision models. With this large scale, it gives strong increases to zero-shot transfer and robustness.
Read MorePaper
Build an Automated Cross-Domain Question Answering System
Question Answering models are often used to automate the response to human questions by leveraging a knowledge base. My team at Stanford aims to build a robust question answering system that works across datasets from multiple domains. We explore two transformer-based Sparsely-Gated Mixture-of-Experts architectures and conduct an extensive ablation study to reach the best performance.
Read MoreData Validation for Machine Learning - Paper Reading Note
This paper reminds me of many time where our model in production perform strangely, so engineers have to spend hours investigate root causes and roll back or push for fixes. Lots of late night works as result of such mistakes. I agree with this paper that such data validation systems, if implemented correctly, can really help save significant amount of engineer hours by catching important errors proactively and diagnose model errors more efficiently.
Read More