{"id":34133,"date":"2023-12-19T02:59:00","date_gmt":"2023-12-19T10:59:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=34133"},"modified":"2023-12-20T20:28:10","modified_gmt":"2023-12-21T04:28:10","slug":"insidebigdata-ai-news-briefs-bulletin-board","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/12\/19\/insidebigdata-ai-news-briefs-bulletin-board\/","title":{"rendered":"insideBIGDATA AI News Briefs Bulletin Board"},"content":{"rendered":"\n

Welcome insideBIGDATA AI News Briefs Bulletin Board, our timely new feature bringing you the latest industry insights and perspectives surrounding the field of AI including deep learning, large language models, generative AI, and transformers. We\u2019re working tirelessly to dig up the most timely and curious tidbits underlying the day\u2019s most popular technologies. We know this field is advancing rapidly and we want to bring you a regular resource to keep you informed and state-of-the-art. The news bites are constantly being added in reverse date order (most recent on top). With our bulletin board you can check back often to see what’s happening in our rapidly accelerating industry. Click HERE<\/a> to check out previous “AI News Briefs” round-ups. <\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

[12\/20\/2023] OpenAI is expanding its internal safety processes<\/a> to fend off the threat of harmful AI. A new \u201csafety advisory group\u201d will sit above the technical teams and make recommendations to leadership, and the board has been granted veto power \u2014 of course, whether it will actually use it is another question entirely.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

[12\/19\/2023] Salesforce announced<\/a> it will start making available Einstein Copilot, its new all-purpose conversational AI assistant for the office. Salesforce aims to enhance its users\u2019 customer service options and offer a new level of efficiency to sales and marketing efforts via a new interface and customizable AI models. AI-powered CRM systems like this will enable Salesforce clients to create more personalized and effective interactions without extensive work developing their own AI models. Salesforce is expecting this product will help streamline the integration of one of its key advantages: the silos of enterprise data its clients maintain on the platform. Ultimately, every so-called data aggregator like Salesforce will need to adapt to this trend to compete with hyperscalers like Microsoft and Google.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n
\n
\"\"<\/figure><\/div>\n\n\n

[12\/19\/2023] OpenAI Guide to Prompt Engineering<\/a>: The guide shares 6 strategies and tactics for getting better results from LLMs like GPT-4. The methods described can sometimes be deployed in combination for greater effect. The company encourages experimentation to find the methods that work best for you. Some of the examples demonstrated in the guide currently work only with the most capable model, gpt-4<\/code>. In general, if you find that a model fails at a task and a more capable model is available, it’s often worth trying again with the more capable model.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n
\n
\"\"<\/figure><\/div>\n\n\n

[12\/17\/2023] NEW from Google DeepMind! FunSearch: Making new discoveries in mathematical sciences using Large Language Model<\/a> – Google DeepMind used FunSearch, a unique LLM framework, to solve a famous unsolved problem in pure mathematics – the cap set problem using a unique pairing of a pre-trained LLM with an automated evaluator for iterative solution development. FunSearch is built on a modified version of Google’s PaLM 2, termed Codey, optimized for code generation. It fills in missing solution components in a Python-sketched problem. The cap set problem involves finding the largest set of points in a high-dimensional grid where no three points align linearly. It’s a complex issue representing a broader class of problems in extremal combinatorics. The approach involved pairing a pre-trained LLM with an evaluator for program generation and refinement. FunSearch first generates a range of potential solutions. The evaluator then rigorously filters these solutions, retaining only the most accurate and viable ones. This process iteratively refines the solutions, enhancing their reliability and applicability. FunSearch outperformed existing computational methods. Access the research paper HERE<\/a>. <\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n
\n
\"\"<\/figure><\/div>\n\n\n

[12\/17\/2023] LLM360<\/a> introduced Amber-7B and Crystalcoder-7B LLMs<\/a>, offering full transparency in Large Language Model (LLM) training with comprehensive open-source release including training data and code. Existing LLMs lack transparency in training processes, limiting the AI community’s ability to assess reliability, biases, and replicability. This obscurity hinders collaborative progress and thorough understanding of LLM behaviors. LLM360 tackles this by releasing two models with all training components – including 1.3T and 1.4T token datasets, training code, intermediate checkpoints, and detailed logs. Training employs AdamW optimizer, mixed-precision techniques, and thorough data mix analysis for nuanced pre-training. Amber-7B and CrystalCoder-7B demonstrate robust performance on benchmarks like ARC and MMLU. Specifics include training on diverse datasets (e.g., RefinedWeb, StarCoder), achieving 582.4k tokens per second throughput, and detailed analysis of model behaviors like memorization across training stages. Download the research paper: “LLM360: Towards Fully Transparent Open-Source LLMs<\/a>.” LLM 360, a collaboration between Petuum, MBZUAI and Cerebras, is dedicated to advancing the field of AI by providing comprehensive access to large language models.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

[12\/17\/2023] COOL PAPER ALERT! Weight subcloning: direct initialization of transformers using larger pretrained ones <\/strong><\/p>\n\n\n\n

Paper page: <\/strong>https:\/\/huggingface.co\/papers\/2312.09299<\/a><\/p>\n\n\n\n

Abstract: <\/strong>Training large transformer models from scratch for a target task requires lots of data and is computationally demanding. The usual practice of transfer learning overcomes this challenge by initializing the model with weights of a pretrained model of the same size and specification to increase the convergence and training speed. However, what if no pretrained model of the required size is available? In this paper, we introduce a simple yet effective technique to transfer the knowledge of a pretrained model to smaller variants. Our approach called weight subcloning expedites the training of scaled-down transformers by initializing their weights from larger pretrained models. Weight subcloning involves an operation on the pretrained model to obtain the equivalent initialized scaled-down model. It consists of two key steps: first, we introduce neuron importance ranking to decrease the embedding dimension per layer in the pretrained model. Then, we remove blocks from the transformer model to match the number of layers in the scaled-down network. The result is a network ready to undergo training, which gains significant improvements in training speed compared to random initialization. For instance, we achieve 4x faster training for vision transformers in image classification and language models designed for next token prediction.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n
\n
\"\"<\/figure><\/div>\n\n\n

[12\/16\/2023] Google announced Gemini Pro<\/a> developer access – its advanced artificial intelligence program, Gemini, which was unveiled last week, will now be available in a preview version for users of its AI Studio programming tool and Vertex AI, a fully managed programming tool for enterprises running on Google Cloud. Gemini will also be integrated into Duet AI, Google’s AI-enhanced coding tool, in the coming weeks. The announcement highlighted Gemini’s training on Google’s custom AI chip, the Tensor Processing Unit (TPU), and the release of TPU v5p, offering four times the performance of the existing v4 chips.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n
\n
\"\"<\/figure><\/div>\n\n\n

[12\/16\/2023] OpenAI Partnership with Axel Springer<\/a> to deepen beneficial use of AI in journalism. The detail allows ChatGPT to summarize its current articles, including gated content. The deal is not exclusive and spans several years, enabling ChatGPT users worldwide to access summaries of global news from Axel Springer’s brands, with links to full articles for transparency purposes. The alignment contrasts with other media companies like CNN, the New York Times, and Disney, who have restricted their content from AI data scrapers. This collaboration is the first of its kind and aims to explore AI-enabled journalism and its potential to enhance journalism’s quality and business model.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

[12\/15\/2023] The Mistral\u2019s Mixtral 8x7B Instruct large language model<\/a> (LLM) is now available on the OctoAI<\/a> Text Gen Solution. Users can benefit from competitive quality to GPT 3.5, the flexibility of open source software, and a 4x lower price per token than GPT 3.5. Details released by Mistral AI this week provided confirmation that Mixtral implements a sparse Mixture of Experts (MoE) architecture, and provided competitive comparisons that showed Mixtral outperforming both Llama 2 70B and GPT 3.5 in several LLM benchmarks. MoE models use conditional computing to limit the number of parameters used in generating each token, lowering the computational needs for training and inference. <\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

[12\/15\/2023] Meet Natasha at www.builder.ai\/natasha<\/a> and see how this AI slowly starts to evolve into a contextual companion across the journey and beyond. Developed by Builder.ai\u00ae<\/sup>, an AI-powered composable software platform for every idea and company on the planet. The AI-powered assembly line fuses together Lego-like reusable features, using Building Blocks\u2122 automation to reduce human effort, leveraging a verified network of experts to vastly extend development capabilities, and producing apps at an exceptionally high success rate that are multitudes cheaper and faster than traditional software development.<\/p>\n\n\n\n

\n