Voices | Why (and how) to use LLMs for your business needs

Hear from Will Sandys, Senior Research & Development Consultant at Amplifi, about how to securely use LLMs in your organization to boost performance and unlock new business potential. Learn easy techniques for integrating LLMs with your internal knowledge base, all while keeping privacy intact.

Large Language Models (LLMs) have been the talk of the industry with promises of transforming organizations and the way we all work. Many of the most well-known models, such as ChatGPT, are cheap and easy to use. Yet they are limited to answering questions based upon the information contained within their initial training datasets. Fun to use, but not scalable or fine-tuned to work for your organization. Additionally, there are privacy concerns for businesses intending to share sensitive data with companies that have opaque terms and conditions.

So, how can you employ techniques to overcome these limitations and securely implement LLMs within your organization? That’s what I’d like to talk you through in this article. When implemented securely, LLMs can access an organization's internal knowledge base - including documents, databases, or any relevant data source - to answer questions insightfully and in an effective tone of voice. This approach, known as "retrieval augmented generation" (RAG), unlocks the business potential of these models, enabling deeper interactions with customers or employees. This article will aim to give you an understanding of how to utilize LLMs for your business, how to enhance their performance to maximize potential benefits, and how data management experts like us at Amplifi can offer customized support and guidance, tailored to your specific needs.

I’ll guide you through several practical ideas that businesses can use with LLMs and provide detailed strategies on the best methods to initiate their implementation effectively. This will help ensure you're well-equipped to start integrating these advanced technologies into your operations smoothly.

Picture 1 1

Setting up for success with LLMs

The application of AI in scenarios like developer tools and employee self-service scenarios vividly demonstrates how machine learning and natural language processing can transform workplace efficiency. By handling routine inquiries and providing rapid responses, AI allows for human talent to be allocated where it's most needed—tackling creative challenges and strategic planning.

LLM Picture 2

Use Case: Enhancing employee self-service with AI-powered HR tools

AI has the potential to significantly transform the landscape of human resources by increasing the efficiency of recruitment processes, improving the onboarding experience, and effectively monitoring employee satisfaction. A typically laborious task recruiters must perform is to objectively assess, summarize and quantify the overall strengths and weaknesses of the applicant cohort for a job advert. In this use case, by providing the received CVs/resumes as context, an LLM would be able to determine what proportion of candidates meet certain criteria (ranging from soft skills to programming in a certain language).

Not only could this be used to power an insightful dashboard that visually demonstrates the status of a recruitment campaign, but also suggestions for altering a job description advert could be made dynamically, whenever it seems the applicants’ skills-mix is not as desired. Similarly, it could suggest interesting questions for individual candidates at interview if a particular skills gap is identified. This would empower the HR team to spend their time on higher-value tasks, such as relationship building. Furthermore, by optimizing prompts or fine-tuning the LLM, it could ensure there is consistency in assessing candidates across all members of the recruitment team.

LLM Picture 3

Use Case: Streamlining developer queries with AI-enhanced documentation access

Developers frequently encounter the need to integrate various APIs into their projects, which requires precise knowledge of how these APIs function. A common question might be related to the type of HTTP method an API uses, such as, "What HTTP Method does the New Sale API use?"

Traditionally, this would involve a time-consuming search through extensive API documentation. However, the integration of LLMs could speed up this process. By providing the API documentation directly to an LLM, developers can receive instant, accurate answers. For example, when inquiring about the New Sale API, the LLM can quickly interpret the documentation and confirm that the API requires the PUT method for new sales as demonstrated by the use case graphic. This interaction not only saves valuable time but also allows developers to stay in the flow of their work, enhancing productivity.

Some top tips on how to get started on implementing LLMs

There are three main approaches in creating an AI for your business – Retrieval-Augmented Generation (RAG), fine-tuning existing models, and developing new models. RAG combines retrieval systems with generative models, offering quick and efficient solutions for many use cases. Fine-tuning enhances pre-trained models for specific tasks, while developing new models provides custom solutions but is resource-intensive. We will focus on RAG for the rest of this article, as it provides quick value across a wide range of use cases.

To give you a steer on how to approach implementing LLMs with your organization, a few of my initial suggestions would be:

  • Understand embeddings: Learn how embeddings capture semantic meanings in text, facilitating tasks like semantic search and customer analysis. I provide a detailed explanation of embeddings below.
  • Identify applications: Pinpoint areas for LLM use, such as AI-powered HR tools for quicker employee queries and AI-enhanced tools for developer efficiency.
  • Prepare for integration: Ensure the right technical infrastructure and robust data management practices are in place, while adhering to ethical standards.
  • Implement and monitor: Start with pilot programs, adjust based on feedback, and continuously update the models.
  • Train your team: Educate employees on effectively using these new AI tools.
  • Evaluate and iterate: Regularly assess the impact and refine solutions to improve outcomes

Mastering Embeddings and Prompt Engineering

A prompt is the question you ask the LLM (each model may have slight nuances in the optimal way to structure them). The current generation of models all have an upper limit to the length of prompt that can be supplied, and their performance may degrade the longer the prompt. Therefore, we must use an intelligent way to pre-filter our documents, to provide meaningful context to the LLM rather than blindly pasting all the text available from our document corpus. To perform this filtering, we utilize embeddings.

Embeddings are powerful tools for building AI systems with domain-specific knowledge. They are numerical ways of representing real-world text - similar to how geographical coordinates (latitude and longitude) map specific locations on Earth, embeddings provide a way to represent the meanings of words in a structured format. This allows us to measure and compare the meanings of different words within a high-dimensional space, making it easier to analyze and understand textual data.

By converting all our documents to embeddings in advance, when a user asks a question, we can rank the documents in terms of their similarity to the question. This allows us to filter out any documents that will not provide meaningful additional context for the prompt. The pipeline below demonstrates this principle.

Enhancing LLM Precision with Document Chunking

Our documents may be long and 99% of their content irrelevant to bring back errors, or vague information without answering the question at hand. For instance, a single clause within a long legal document may be all that is relevant. One effective technique to maximize the precision of our prompts is “document chunking”. As discussed, LLMs often struggle with long documents due to memory constraints and context dilution. Chunking—dividing a lengthy text into smaller parts—ensures more manageable processing and helps maintain contextual accuracy.

Picture 4

There are several approaches to chunking. The simplest is fixed-size chunking, where a document is uniformly divided into segments of a specific word or character count. However, this method can sometimes split important contextual information.

A more nuanced approach is semantic chunking, where texts are divided based on natural divisions like chapters or topics. This technique respects the document's inherent structure, ensuring that each chunk retains a complete idea or concept. However, it requires more sophisticated processing to identify these natural divisions accurately.

In many cases, a hybrid approach proves most effective. It combines fixed-size and semantic chunking, balancing the ease of uniform segmentation with the need to preserve meaning.

Implementing chunking involves selecting the right tools. Many text processing libraries offer basic chunking functionalities, but for semantic chunking, more advanced natural language processing (NLP) tools may be necessary. Integrating chunking into your LLM workflow involves testing to find the optimal chunk size and approach for your specific needs.

While chunking greatly enhances the precision of LLM outputs, it's not without challenges. One major issue is ensuring that the context is not lost across chunks. Overlapping text segments or incorporating summary sentences at the start or end of chunks can help maintain continuity but would need human editing and/or intervention to ensure the content flows as intended.In the example below, a prompt has been created using a chunking approach. The user’s question (in pink) is combined with specific and relevant quotations drawn from the company’s internal documents (in yellow), so the LLM can properly answer the question.

Picture 5

A word of caution...

When working with current models, it’s important to remember their limitations due to their probabilistic nature. This means that they can produce different outcomes from the same input. Therefore, they may not be reliable for tasks that require outputs with precise formatting. Experiment with requesting responses in a specific and structured format (such as JSON or XML)

How to approach choosing an LLM engine for your organization

Earlier in the article, I shared a few top tips on getting started with LLMs, but this section is worth significant consideration before jumping right in. When navigating the world of AI language models, adopting a considered approach that aligns with your organization's specific needs is crucial. Before integrating AI, you should carefully identify and consider your organization’s unique requirements.

Key areas of focus could include understanding the capabilities of the models, assessing compatibility with your current systems, and determining the impact on your workflow.

Identifying your requirements

  • Use Case Clarity
    What's your goal? Whether it's customer support, classification, defining the primary function of the LLM is crucial.
  • Domain-Specific Data
    Consider the nature of your data. If it's specialized, like healthcare or finance, opting for a model trained on similar data can yield better results.
  • Accuracy Needs
    Balance the need for precision with the capabilities of larger models, which generally offer higher accuracy.
  • Inference Speed
    For time-sensitive applications, smaller, faster models might be more appropriate.
  • Scalability
    Estimate your user base and query volume. Large models can be costly to scale.
  • Privacy Needs
    Utilizing vendors, such as Microsoft’s suite of AI tools (such as Prompt Flow and Copilot) may allow for rapid development whereas deploying a fully private and self-hosted LLM and embedding agent is possible but can be complex, time-consuming, and more expensive.
  • Budgetary Considerations
    Costs vary with usage, model size, and query number. Find a balance that works for you.
  • Ethical Concerns
    Assess potential biases and risks, especially in sensitive applications.

If you’re looking to take the next step with AI or LLMs, Amplifi are well positioned to assist you at any stage of your journey. LLMs are not a simple off the shelf solution, and not one size fits all, so we’re here to help fine-tune exactly what your organization’s requirements are, whether that be categorization, general set-up, or looking at your specific applications/integrations. You can reach out to myself, or any of our data experts here.

Mike Evans, Chief Innovation Officer at Amplifi has also produced a new guide, 6 expert tips for driving value with AI – you can download it below.

Download guide | 6 expert tips for driving business value with AI

Amplifi Driving Business Value with AI Guide Mockup

About the author

Will Sandys is Senior Research & Development Consultant at Amplifi. Being a seasoned expert in data analysis, data modelling, and data quality profiling, with extensive experience working with Amplifi and Big 4 consultancy clients, Will combines a deep technical skillset with a thorough understanding of the business context of data and its significance.

Will has led the technical and data modelling elements of over many data migrations and now collaborates with our Chief Innovation Officer to deliver innovative solutions to Amplifi clients, including cutting-edge Machine Learning and AI solutions. His expertise in LLMs and AI positions him at the forefront of developing advanced, domain-specific AI systems, making complex topics accessible and actionable for businesses.

Fill out the form here to arrange a meeting with Will, or to ask any specific questions.

Will Blog Image