A New York business chatbot is sending out some particularly bad information
AI has the answer – just the wrong one
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
An AIchatbotreleased by the New York City government designed to assist business owners in accessing information has come under scrutiny for sharing inaccurate and misleading guidance.
A report byThe Markup,co-published with local nonprofit newsroomsDocumentedandThe City, reveal multiple instances where the chatbot provided wrong advice about legal obligations.
For example, the AI chatbot claimed that bosses could accept workers’ tips and that landlords are allowed to discriminate based on source of income – both wrong pieces of advice.
Chatbot fail?
Launched in October 2023 by Mayor Adams’s administration as an extension of the MyCity portal, the chatbot, described as “a one-stop shop for city services and benefits,” is powered byMicrosoft’s Azure services. Despite its intention to serve as a reliable source of information sources directly from the city government’s websites, the pilot program has been found to generate flawed responses.
One example given byThe Markupsees the chatbot asserting that businesses could operate as cashless establishments, despite the New York City’s 2020 ban on such practices.
Responding to the report, Leslie Brown, spokesperson for the NYC Office of Technology and Innovation, acknowledged the chatbot’s imperfections, emphasizing ongoing efforts to refine theAI tool:
“In line with the city’s key principles of reliability and transparency around AI, the site informs users the clearly marked pilot beta product should only be used for business-related content, tells users there are potential risks, and encourages them via disclaimer to both double-check its responses with the provided links and not use them as a substitute for professional advice.”
Are you a pro? Subscribe to our newsletter
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
After a months-long honeymoon period, the cracks are beginning to show as businesses and government agencies start to question the reliability, safety and security of artificial intelligence, with many imposing bans and others introducing strict regulations.
More from TechRadar Pro
With several years’ experience freelancing in tech and automotive circles, Craig’s specific interests lie in technology that is designed to better our lives, including AI and ML, productivity aids, and smart fitness. He is also passionate about cars and the decarbonisation of personal transportation. As an avid bargain-hunter, you can be sure that any deal Craig finds is top value!
LG Electronics sets ambitious B2B revenue goal to offset declining consumer demand
New fanless cooling technology enhances energy efficiency for AI workloads by achieving a 90% reduction in cooling power consumption
Phishing attacks surge in 2024 as cybercriminals adopt AI tools and multi-channel tactics
