High Dimensional and Complex Spectrometric Data Analysis of an Organic Compound using Large Multimodal Models and Chained Outputs
September 12, 2024
Kevin Kawchak
CEO ChemicalQDevice
kevink@chemicalqdevice.com
Large Multimodal Models (LMMs) possess the ability to analyze chemical spectra of an organic compound using state of the art conversational AI. These outputs can then be chained together and introduced as a text input for other LLMs or LMMs to predict the compound name. Here, a challenging 15 carbon molecule problem with 13 complex and high dimensional chemical spectra were analyzed as images by unmodified versions of Claude 3.5 Sonnet and OpenAI ChatGPT-4o models. ScholarGPT judged the responses across the 13 spectra with an average score of 9.01/10, and the highest response scores per individual spectra for 3.5 Sonnet or GPT-4o were used as the text-based chain. For Part B, the chain was then combined with two different prompt formats and the molecular formula to 8 different LMMs or LLMs which produced new compound predictions. 3.5 Sonnet had the highest proficiency in utilizing the formula simultaneously with complex data for three identical compound generations across two prompts, but was likely limited by the quality regarding the chain of 13, primarily with data from 6 2D NMR Spectra. 3.5 Sonnet's compound prediction was then further improved in Part C by utilizing manual chained explanations of the spectra by the author to yield what is believed to be the correct structure with stereochemistry to the unknown problem. To the author's best knowledge, this is the first LMM to generate the C15H22O2 drug compound derivative (S)-ibuprofen ethylester using high dimensional data from 13 detailed spectra. The purpose of this study was to utilize cutting edge natural language processing techniques to evaluate an advanced chemical structure consisting of IR, 1H-NMR, 13C-NMR, DEPT-NMR, GCOSY60, GTOCSY, GHMQC, GHMBC, GNOESY, and expanded views of spectra. Manuscript
LMM Spectrometric Determination of an Organic Compound
August 26, 2024
Kevin Kawchak
CEO ChemicalQDevice
kevink@chemicalqdevice.com
Many machine learning models used in academia and industry that identify organic compounds typically lack the ability to converse over prompts and results, and also require expertise across a number of steps to obtain answers. The purpose of this study was primarily to gain insight into the advantages of current unmodified state of the art Large Multimodal Models (LMMs) across several prompts containing multiple spectra of varying difficulty to evaluate the impact of training data, reasoning, and speed. These readily available and easy to use software for the identification of an organic compound based on a molecular formula and spectra were found to be reproducible across three similar LMMs. To the author's best knowledge, this marks the first time that three GPT variants were each able to correctly identify the organic compound quinoline using a variety of different spectroscopic images. The results were obtained using a 2-step process consisting of a) Uploading high resolution spectral images, and b) Submitting a text prompt with the images that requested a compound determination. The main findings were that 1) Four LMMs provided rationale step-by-step interpretations of 1H-NMR, 13C-NMR, and 3 DEPT-NMR spectra from Prompt A, 2) Three of these LMMs, led by a GPT-5 preview model, combined these interpretations into the correct chemical structure with Prompt A, and 3) Two of these LMMs achieved a top score of 5/5 for also generating sequential explanations reflecting the order of the provided spectra along with most of the correct spectral and molecular formula explanations. Manuscript, Seminar
LMM Chemical Research with Document Retrieval
Kevin Kawchak
Chief Executive Officer
ChemicalQDevice
San Diego, CA
August 12, 2024
kevink@chemicalqdevice.com
Chemical research is more effectively progressed using Large Multimodal Models (LMMs) combined with Document Retrieval and recently published literature. The methods described here illustrate significant strides over previously tested Large Language Model (LLM) multi-document workflows for characterization assistance and generating new reactions. Here, 3.5 Sonnet, ScholarGPT, and ChatGPT 4o LMMs processed either 5 images or 5 supplementary documents from leading 2024 journals. Each of the three models performed inference on a detailed prompt to produce a response that included context from attachments. In addition, the LMMs were not provided with which of the 5 files contained the answer. The main findings were that 3.5 Sonnet had an average score of 9.8 for images, while two judges awarded high scores to ChatGPT 4o (9.7, 9.4) and ScholarGPT (9.5, 9.4) for document analysis. Judging was performed by a human evaluator for the image uploads, with document processing evaluated by Llama 3.1 405B and Nemotron 4 340B LLMs which correlated well and improved explainability. Highlights include 3.5 Sonnet's ability to interpret a Two-dimensional Nuclear Magnetic Resonance (2D NMR) spectrum accurately, along with Judge Llama 3.1's ability to provide consistent formatted scores with explanations. The results shown here help illustrate AI's continued revitalization of the established chemical research field. Manuscript, Seminar