There is new king in the NLP models! The Chinchilla NLP model beats GPT-3 and Gopher by optimizing model size. The Chinchilla is trained less parameters and more tokens. Let’s familiarize with it! Introduction The GPT-3 was major breakthrough in 2020. It triggered wave of large NLP models. I imagined…

The GPT-3 introduced new features called Insert and Edit. The features improve contextual knowledge and allows using GPT-3 in new ways. — The GPT-3: Edit The Edit is a feature, which instructs the way we want the GPT-3 to change our text. It allows quick modification of texts using written instruction. The new Edit-endpoint requires only one mandatory parameter called: Instruction. It defines the way the input-parameter needs to change. Let’s review…

The Video PreTraining (VPT) learns from unlabeled video to make sequential decisions. — Introduction The researchers created an agent capable of playing Minecraft at human-level. The pretraining required 70k hours of video. The VideoPretraining (VPT) technique is here! The technique is generic for any domain. It is not only limited to playing Minecraft. …

Google’s LIMoE scales efficiently with State-of-the-Art performance in image classification. — Introduction LIMoE is a multimodal image image classifier. It is Sparsely-activated. 5.6B parameters, 675M parameters per token Sparsely-activated Mixture of Experts (MoE) model Multimodal contrastive learning model. Transformer-based model The model offers State-of-the-Art level results, while scaling compute in efficient manner.

New neural theorem proving techniques lead towards AGI. — Introduction Mathematics is a challenge to AI research due to (a) infinite action space & (b) lack of self-play. Yet, the AGI implies the ability for mathematical reasoning. Lean, Metamath, and Equations are environments used for proving formal mathematical theorems. Such tools offer high-level tactics to apply to the problem…

Google Brains releases text-to-image generator with State-of-the-Art performance. — Introduction Imagen is the new State-of-the-Art text in text-to-image generation. It is better than GLIDE or DALL·E 2. It achieves FID score of 7.21 in zero-shot learning with the COCO dataset. Imagen is 4.6B parameter text-encoder based on T5-XXL. The text encoder uses text only corpus, not image-text pairs. It…

Gato is capable of performing +600 tasks — play video games, chat, caption images and move robotic arms in real-world & simulated environments. — Introduction Gato manages multi-modal, multi-task and multi-environment — within single, generalist agent. DeepMind trained this generalist with 604 different tasks. It outperformed expert level in 450 of them — at least 50% of time. The model is transformative in terms of diversity of tasks it manages to perform. It can…