New York City hospitals drop Palantir as controversial AI firm expands in UK
In a significant move, New York City’s public hospital system has announced that it will not renew its contract with Palantir Technologies, a company known for its controversial role in data analytics and artificial intelligence. This decision comes amidst growing scrutiny of Palantir’s operations, particularly regarding its dealings with the UK’s National Health Service (NHS).
Details of the Contract with Palantir
Dr. Mitchell Katz, president of NYC Health + Hospitals, testified before the New York City Council that the contract with Palantir, which focuses on optimizing revenue collection from insurance claims, is set to expire in October. Katz emphasized that the agreement was always intended to be short-term and included safeguards to prevent any sharing of information with U.S. Immigration and Customs Enforcement (ICE).
According to documents shared with The Guardian, NYC Health + Hospitals has paid Palantir nearly $4 million since November 2023. The contract allowed Palantir to review patient health notes and assist the hospital in claiming public benefits, including Medicaid. Notably, the agreement included a clause permitting Palantir to “de-identify” patient data for purposes beyond research, raising significant privacy concerns.
Transitioning to In-House Systems
In response to the growing concerns, NYC Health + Hospitals has stated that it will transition to systems developed entirely in-house, ensuring that no data will be shared with Palantir or its applications after the contract ends. The agency clarified that its use of Palantir technology was strictly limited to revenue cycle optimization, aimed at closing gaps between services delivered and charges captured.
A spokesperson for Palantir defended the company, stating that it does not own customer data and that each customer environment is protected against unauthorized access through robust security controls.
Palantir’s Expansion in the UK
As New York City’s hospital system prepares to cut ties with Palantir, the company is simultaneously expanding its presence in the UK. Palantir is under scrutiny for its £330 million contract with the NHS, which has faced backlash from health officials concerned about privacy issues. Despite efforts by Labour leader Keir Starmer to expedite the deployment of Palantir’s data systems, less than half of the UK’s health authorities had begun using the technology as of last summer.
Medact, a health justice charity, has raised alarms about the potential for Palantir’s software to facilitate “data-driven state abuses of power,” including the possibility of U.S.-style ICE raids. Palantir has denied these allegations, asserting that such uses would be illegal and a breach of contract.
Controversial Contracts and Data Privacy Concerns
Palantir’s influence in the UK is growing, despite opposition from activists and some lawmakers. Recently, the Financial Conduct Authority awarded Palantir a contract to investigate internal intelligence data to combat financial crime, prompting calls from Liberal Democrats for a government investigation into this agreement. Starmer has acknowledged concerns about the UK’s reliance on American technology firms but has expressed a preference for enhancing domestic capabilities.
Privacy advocates have voiced concerns over Palantir’s access to de-identified patient data within the NHS. De-identified data refers to information stripped of personal identifiers, but experts warn that advancements in AI make re-identification increasingly feasible. Sharona Hoffman, a law professor, noted that de-identification no longer guarantees anonymity.
Ari Ezra Waldman, another law professor, emphasized the risks associated with companies like Palantir collecting information on vulnerable populations, particularly when contracts allow for data use beyond research purposes. He expressed concerns that the government may lack sufficient power to negotiate effectively with Palantir.
Conclusion
The decision by New York City hospitals to discontinue their contract with Palantir reflects growing concerns over data privacy and the ethical implications of using advanced analytics in public health. As Palantir continues to expand its operations in the UK amidst similar scrutiny, the debate over the balance between technological advancement and privacy rights remains at the forefront of public discourse.
Note: This article is based on information available as of October 2023 and aims to provide a comprehensive overview of the ongoing issues surrounding Palantir Technologies.
