Drawing on our experience helping businesses adopt AI responsibly—plus insights from our recent interview with Yoann, our LLM expert—we’ve compiled the critical do’s and don’ts every decision-maker should consider before implementing an LLM.

Do #1: Start with a clear business objective

Why it matters:

Before you rush to integrate the newest model, define the exact problem you expect it to solve. It could be faster customer service response times, automated invoice handling, or reduced manual tasks. Companies that integrate LLMs without a clear goal often wind up with solutions searching for problems—wasting time and money.

Best practices:

• Break down your objective into specific, measurable targets, like “reduce customer service handling times by 30% within one year.”

• If a simpler automation or analytics tool meets your need, deploy that first. AI or not, the solution just needs to work.

Don’t #1: Chase every “revolutionary” LLM without checking fit

Why it matters:

Many version upgrades—ChatGPT 4.0 to 4.5, Claude 3.6 to 3.7, etc.—are incremental or cater to narrow use cases. Although Claude may outperform ChatGPT in coding tasks, ChatGPT excels at general knowledge queries. Adopting each “next big thing” can lead to “technology whiplash,” where your team invests more time upgrading than seeing a return on investment (ROI).

Key considerations:

• Evaluate impact on your core use case. If the update merely offers minor gains or specialized features you won’t fully utilize, wait for feedback before switching.

• Match the model to your tasks. If your priority is coding assistance, a code-focused LLM like Claude might outperform a general-purpose chatbot. For analytics or data extraction, opt for models built with those strengths in mind.

• Stay strategic. The LLM market evolves fast. A major upgrade could arrive soon, so don’t feel compelled to change models unless you see clear, measurable benefits.

Yoann: If a new model is making headlines, it’s usually more about marketing than a real breakthrough. If your current model is working well for your organization, the next version is unlikely to be a game-changer. Unless there’s a significant leap in performance or cost-effectiveness, it’s often wiser to hold off rather than rushing to upgrade every time.

Do #2: Always keep a human in the loop

Why it matters:

LLMs often respond with great confidence—even when they’re wrong. For customer-facing applications, especially in regulated environments like healthcare or finance, an unchecked bot can produce misleading or inaccurate responses. Having employees or subject-matter experts supervise high-stakes interactions reduces these errors.

Best practices:

• Add escalation rules: if the model is “uncertain” or if the request involves a high-value transaction, direct it to a human agent.

• Regularly review conversation logs and outcomes to ensure quality control.

Don’t #2: Give client-facing chatbots free rein

Why it matters:

Even the best LLMs can be vulnerable to hallucinations or “prompt injection,” where a malicious user manipulates the conversation so the model grants unauthorized actions or shares restricted information. One anecdote involves Air Canada’s customer support chatbot, which a savvy user fooled into granting a nonstandard refund.

Best practices:

• Use strict role-based permissions. Even if the LLM “believes” it can authorize refunds, it should have no real authority to do so without a second approval.

• Monitor for unusual requests or suspicious language that might signal an injection attempt.

Yoann: The technology isn’t quite there yet, and most companies aren’t willing to take that risk. If internal-facing chatbot users can confidently say, ‘It never makes mistakes and fully understands context,’ then you could consider making the bot client-facing. But for now, I always recommend keeping a human in the loop to prevent costly errors.

Do #3: Deploy internal-facing chatbots first

Why it matters:

Internal-facing chatbots can streamline operations, reduce routine tasks, and boost employee productivity by offering quick answers to internal questions. Whether it’s HR inquiries or IT support, chatbots can save time and free employees to handle more valuable work.

Additionally, launching a chatbot internally first lets your organization test its reliability, accuracy, and overall effectiveness before introducing it to customers. This approach lowers risk while giving you a chance to refine your chatbot’s performance.

Best practices:

• Map out key automation opportunities. Identify repetitive tasks, such as HR or IT requests, that can benefit from chatbot integration.

• Ensure seamless system connections. Integrate your chatbot with the relevant internal databases or intranet sites, so it can provide accurate, real-time answers.

• Gather employee feedback. Regular input from internal users helps you improve chatbot features, laying the groundwork for a future customer-facing chatbot.

Don’t #3: Overlook cost and infrastructure demands

Why it matters:

Running LLMs can require considerable resources. Hosting an open-source model locally might give you more control but demands additional hardware. On the other hand, using external APIs can lead to unpredictable monthly bills if usage spikes.

Best practices:

• Do a cost-benefit analysis when comparing self-hosting vs. a cloud-based solution.

• For modest usage, an API subscription might be cheaper. For large volumes, a self-hosted model might pay off—though you must maintain the hardware.

Do #4: Align your data strategy with your LLM deployment

Why it matters:

An important takeaway from our interview is “data hygiene.” Clean, well-organized, and centralized data significantly improves any AI system’s reliability. Messy or incomplete data can cause erratic or outright harmful outputs. Plan for sensitive information from the start, as many external APIs (including ChatGPT’s or Deepseek’s) offer fewer security guarantees than in-house tools.

Best practices:

• Invest in data governance and master data management (MDM) before you attempt a large-scale LLM roll-out.

• Check data sources for missing values, inconsistent formats, or outdated records.

• When dealing with confidential data, ensure security and compliance: implement strong access controls, encryption, and regular audits.

Conclusion

Deployed wisely, LLMs can deliver major benefits—faster service, streamlined processes, and lower costs. Yet, as the Air Canada chatbot incident shows, rushing or neglecting safeguards can be risky.

For decision makers, due diligence is the key:

• Frequently reassess whether the latest “breakthrough” model genuinely suits your strategies.

• Pinpoint a clear use case and ROI goals from the start.

• Match the approach to your data’s sensitivity.

• Keep people in the loop, especially for critical or high-value processes.

• Start with internal chatbots before going customer-facing.

• Strengthen data hygiene and master data management.

Following these do’s and don’ts helps you harness LLM technology while avoiding the pitfalls that can cause expensive refunds and reputational damage.

Ready to deploy LLMs responsibly?

Have more questions about data governance, model selection, or safe deployment practices? Contact us to discuss how best to integrate LLMs securely and seamlessly into your organization.