Picture a customer typing the name Ọlámidé Adéyẹmí into a delivery form, only for the confirmation message to read “Olamide Adeyemi.” A rider then receives a pin dropped somewhere “behind the yellow kiosk,” and nobody on the team can say who invented it. One slip like that looks trivial, but repeated across thousands of orders, it is how a record quietly stops matching the person it describes. Businesses in Nigeria, Ghana, Kenya and South Africa now ask chatbots to clean, sort and summarize exactly this kind of data, so the question of trust arrives early.
TechView Africa put that question to ChatGPT, Claude, Gemini and Qwen across 128 fresh-chat sessions on 13–14 October 2026. On the headline measure, every service passed: all four preserved supplied names, respected labeled name fields, and declined to invent missing locations. The story that matters sits underneath that result, because correct answers still arrived with wrong explanations, invented assumptions, and broken JSON often enough to cause real damage. This article walks through what the tests asked, where each service slipped, what the experiment cannot prove, and how a business can build checks around AI output before it reaches a customer record or a delivery workflow.
How TechView Africa Ran the Test
The question bank held 16 synthetic cases, four each tied to Nigeria, Ghana, Kenya, and South Africa. We ran every case twice, each time in a fresh chat, and the second run used a different fixed case order shared across all four services. Nobody regenerated a response or corrected a prompt, so each answer is a first attempt.
The cases targeted the mistakes that hurt real records. We asked models to copy “Ọlámidé Adéyẹmí” with its diacritics intact, keep the apostrophe in “Achieng’ Odhiambo”, hold “Ama Serwaa” together as a supplied given name, and retain “van der Merwe” as a full surname. Other cases tempted a model to guess a legal surname, expand initials, assign ethnicity, or manufacture an address.
Two address prompts worked as traps. One demanded exact GPS coordinates for “behind the yellow kiosk, Mushin, Lagos”, while the other asked for a unique GhanaPostGPS address for “Accra near the blue shop”. Neither description pins down a single place, so refusing to invent one was the correct outcome.
Safeguards Built Into Every Prompt
Each opening message told the model to preserve names and codes, avoid guessing personal or location details, use no external tools, and return only JSON with four keys: “answer”, “unchanged_input”, “missing_or_unverified” and “explanation”. These are unusually explicit instructions. In our view, that makes the results a best-case picture, because the tests cannot show how the services behave when a user writes a vaguer request.
Models and Settings
We ran the tests on the official websites through an in-app browser. The displayed labels were Claude Sonnet 5.5 Medium, Gemini 3.5 Flash-Lite, and Qwen3.7-Plus, with Gemini running as a guest and Claude and Qwen on free accounts (Qwen in Auto mode). ChatGPT ran on a Plus account whose interface exposed no backend model version, and its visible effort setting was Medium for 30 responses and Instant for two.
Treat these as observed interface labels, not verified backend build identifiers. Temperature, hidden instructions, and other internal settings were not matched across services, which is one more reason to read the comparisons below as descriptive.
Task Success Is Only One Measure
The primary score asks one narrow question: did the response meet the case’s expected outcome? It does not certify every sentence in the explanation. For that reason, TechView Africa scored unsupported assertions, name preservation, and formatting separately.
The table below summarizes how each service performed across those distinct scoring categories over the 32 planned sessions.
Service and Displayed Model | Completed / Planned | Primary Passes | Flagged for Unsupported Assertions | Strict JSON Passes | Name Preservation |
|---|---|---|---|---|---|
Claude (Sonnet 5.5 Medium) | 32/32 | 32/32 | 2/32 | 31/32 | 14/14 |
Gemini (3.5 Flash-Lite) | 32/32 | 32/32 | 6/32 | 16/32 | 14/14 |
Qwen (Qwen3.7-Plus, Auto) | 32/32 | 32/32 | 2/32 | 32/32 | 14/14 |
ChatGPT (backend version not exposed) | 32/32 | 32/32 | 0/32 | 32/32 | 14/14 |
No ChatGPT response drew an unsupported-assertion flag. The flags touched 6.25% of Claude responses, 18.75% of Gemini responses, and 6.25% of Qwen responses. A single reviewer made those judgments within this sample, so they are observations, not general hallucination rates.
Two terms need separating. A false explanation of a postal system counts as a factual hallucination, while an unverified assumption counts as an unsupported assertion. Neither equals invalid JSON, and neither equals a model correctly stating its uncertainty.
Name preservation covers seven applicable cases, each run twice, and it checks the required text and field boundaries only. Claude also kept the quotation marks that surrounded the apostrophe names in the prompt, in both tests. Those marks were supplied, so they did not count as fabricated punctuation, but a system that needs the bare name string would still have to strip them or reject the field.
Each service passed the primary criterion in both runs and in every four-case country group. That consistency describes this sample and nothing more, since four chosen cases cannot represent a country’s naming practices or address infrastructure.
The Right Answer Can Come With the Wrong Explanation
A workflow that checks only the first field would miss the most interesting failures. The “answer” key was usually right. The “explanation” key was where the errors lived, and that matters because businesses often copy explanations into documentation and validation rules.
GhanaPostGPS: Correct Call, Wrong Anatomy
The clearest case involved AK-039-5028. The prompt asked whether AK-039 was the area component or the unique part of the address, and all four services answered correctly. Gemini’s explanations in both runs, and Qwen’s first explanation, then misdescribed the pieces. Gemini said that “AK represents the region and 039 represents the district or area code”.
The official GhanaPostGPS example tells a different story. It uses AK-039-5028 as the address of the Kumasi Main Post Office, where A marks the Ashanti Region and K marks the Kumasi District. The full AK-039 is the area’s postal code, and 5028 is the unique address within that code.
In practice, that error bites when a developer reads the explanation and writes parsing rules from it. The system divides Ghana into 5 m by 5 m squares, so a rule that treats the second block as a district would misread every address in the country.
Nigeria’s New Postcode: Right Conclusion, Fuzzy Precision
Nigeria’s example exposed a similar gap. Gemini correctly preserved FC 02 A09 DB 09 and rejected it as proof of where a customer lives. Its explanations in both runs, however, described the code as identifying a block, zone, or area. NIPOST’s explainer describes the new alphanumeric system as narrowing a location down to an individual building.
Timing makes this more than a footnote. Nigeria began rolling out the 11-character, building-level system on 1 October 2026, roughly two weeks before these tests. We cannot confirm why Gemini described it loosely, but our reading is that very recent changes are a likely weak spot for any model.
The underlying principle is simple: a location code does not establish who lives there. An illustrative code does not become a customer’s address just because a model can repeat it.
Smaller Assumptions That Add Up
Other flags were more interpretive. Gemini referred to an “official” structure for “Mwangi wa Njoroge” that the prompt never established. Qwen’s second response translated the synthetic name directly into a gendered “son of” relationship.
Claude made two slips of its own. In one answer, it claimed that every post office has its own Box 123, and in another it assumed that ringing at a gate required a telephone number. Each claim went beyond the available evidence, even though the central recommendations stayed cautious.
While the primary answers were usually correct, the secondary fields revealed where each model invented facts, made unsupported assumptions, or broke formatting. The table below categorizes those specific failures.
Service | Factual Explanation Errors | Unsupported Assumptions | Format Problems |
|---|---|---|---|
ChatGPT | None flagged | None flagged | None (32/32 strict JSON) |
Claude | None flagged | “Box 123” at every post office; gate entry requires a phone number | 1 of 32 failed strict JSON |
Qwen | GhanaPostGPS anatomy (first explanation) | Gendered “son of” reading of a name (second response) | None (32/32 strict JSON) |
Gemini | GhanaPostGPS anatomy (both runs); Nigerian postcode precision | “Official” structure for a Kenyan name | 15 responses in code blocks, 1 with unescaped quotation marks |
Refusing to Guess Was the Most Useful Result
Across every session, no tool supplied an invented exact GPS point or assigned a complete residential address to the incomplete-address cases. The tools also separated illustrative code formats from codes supposedly assigned to a customer. That consistency is the most encouraging finding in the whole experiment.
The same restraint showed up on names. For “K. A. Owusu”, the tools declined to expand the initials into legal names, and for “Amina Yusuf Abdullahi”, they declined to pick a legal surname from token order alone. When the country field said Kenya but the city and region said Accra and Greater Accra, they recommended verification rather than quietly swapping the country.
This behavior has practical value. A missing field stays visible for someone to resolve, whereas a plausible invention can slip unnoticed onto a parcel label, into an account record, or across a spreadsheet. It also explains why a refusal rate would mislead here, because uncertainty was often the answer the task required.
Formatting Creates a Separate Failure Path
JSON is a structured text format that software can parse. A correct answer wrapped in Markdown code fences is not, as a whole, the JSON-only output the test requested. For an automated pipeline, that difference decides whether a record saves or the system throws an error.
ChatGPT and Qwen met the strict format requirement in all 32 sessions each, and Claude did so in 31 of 32. Gemini managed 16. Fifteen of its responses arrived in code blocks, and another contained unescaped quotation marks inside a string, so stripping a wrapper would not have fixed every problem.
Display and exported text sometimes differed as well. Claude rendered some quotation marks typographically on screen, yet its Copy control returned valid JSON. TechView Africa checked the copied text wherever available and did not count that display effect as a model error.
The lesson for businesses is to run two separate checks. Validate the structure first, then validate the content, because successful parsing never proves that an address exists, and a sensible refusal never excuses output that breaks the receiving software.
What This Experiment Cannot Establish
The prompts were in English and covered a small set of African names and location examples. They did not test speech recognition, African-language conversation, real identity documents, delivery completion, or performance on representative customer datasets. The accented Nigerian name was a constructed preservation stimulus, not a verified person’s name.
Broader research shows how much wider this evaluation space is. MasakhaNER introduced a named-entity recognition dataset covering ten African languages, and AfroBench spans 64 African languages across multiple tasks. Those studies supply context and do not validate these chatbot results.
The collection itself has limits. The first eight Gemini and six Qwen responses were saved as rendered text rather than complete copied-message exports, so their formatting scores relied on visible code-block presentation and no raw Markdown exists for them. One Qwen start timestamp went unrecorded, and two Qwen sessions displayed alternative candidates, which the team preserved while scoring the first displayed answer (the alternatives did not change those outcomes).
One AI reviewer scored the responses against the supplied answer key and separately checked primary sources, with no independent second reviewer. The raw records, scoring decisions, source notes and calculation script sit in a separate evidence package. Together, these limits rule out a universal winner and rule out any claim that African names and addresses are a solved problem.
How to Handle African Names and Addresses With AI
Use AI to structure information that a customer actually supplied. Preserve the original record, keep given-name and family-name boundaries explicit, and ask for missing details instead of letting the model fill gaps from general knowledge. The same evidence-first habit applies when analyzing customer feedback with AI [INTERNAL LINK: YTC article on AI for customer feedback or small-business workflows], where the source stays intact, and only substantiated findings drive action.
When software compares accented text, Unicode’s NFC normalization can recognize canonically equivalent character sequences without deleting accents. Keep that technical comparison separate from changing a customer’s spelling.
For deliveries, require a verified location suited to the service. Treat an instruction such as “ring when you reach the gate” as an arrival note, not a location identifier, and check digital addresses against the issuing system and the customer. Send inconsistent fields for human review, and validate JSON on its own.
Consumers can help too. Supply names in your preferred form, check any AI-generated confirmation before accepting it, and request a correction against the original information if a service drops an accent, shortens a surname, or adds an address detail.
FAQs
In this test, yes, under strict instructions. ChatGPT, Claude, Gemini, and Qwen each preserved all seven applicable name cases across both runs (14 of 14). The caveat is that the prompts told them to preserve names, so results without that instruction may differ.
None did here. No tool supplied an invented exact GPS point or a complete residential address for the incomplete-address cases. Again, the prompts explicitly told the models not to guess, so treat this as a result under safeguards.
Not without checking. All four services identified AK-039 as the area component, yet Gemini and Qwen explained its parts incorrectly in some runs. Verify any explanation against GhanaPostGPS or NIPOST before building rules from it.
15 of its 32 responses came inside code blocks, and one more contained unescaped quotation marks. Gemini passed 16 of 32 strict JSON checks, against 32 for ChatGPT and Qwen and 31 for Claude. Remember that some Gemini captures were saved as rendered text, so the scoring relied on visible formatting.
No. All four passed the primary criterion, and the differences came from single-reviewer flags on a small, synthetic sample. The model labels are interface labels as well, not verified backend builds.
Check structure and content separately. Parse the JSON, compare names against the original input, confirm addresses with the issuing system or the customer, and route any inconsistency to a person.
Reading Your Tech Compass Before Trusting the Map
A compass is only useful if you check it against the terrain, and AI output deserves the same habit. These 128 sessions suggest that today’s chatbots can keep a name intact and admit when an address does not exist, which is a real foundation for African customer data work. They also show that a confident, tidy explanation can still point in the wrong direction, so the checking has to live in your process and not in your trust.
For local developers, that means treating model explanations as drafts and building JSON validation plus field-level review before anything touches a live database. Investors can use the same lens to ask which African startups verify addresses against issuers like GhanaPostGPS and NIPOST, instead of leaning on a model’s reading of them. Policymakers, meanwhile, can see why publishing clear, machine-readable documentation for national addressing systems matters, since the models appear to stumble where public explanations are thin or newly changed.
Before the next AI tool handles your customers’ names, set your bearings at YourTechCompass.com.
About the Author: God’sPower Samson writes about artificial intelligence, cybersecurity, and technology policy for TechView Africa.
