DeepSeek: The Good, the Bad, and the Ugly – Part 2

Last time, I outlined what I believed to be the most important innovation behind DeepSeek that enabled them to train and deploy cutting-edge LLM at a fraction of the cost. Although each of the innovations contributes only a modest efficiency improvement (typically a single-digit multiple), their effects compound when combined, driving huge efficiency gains.

Now that we’ve seen the “Good” side of DeepSeek, let’s examine the opposite side today. We’ve already discussed how AI labs in China must operate under very different constraints, both economically (due to diminishing foreign investment) and technologically (due to chip export restrictions). Beyond these constraints, they must operate under very different security, privacy, safety, and censorship requirements, which are the topics we will discuss today.

It’s important to note that many of DeepSeek’s shortcomings stem from its development within China’s regulatory environment, which imposes strict legal requirements on data handling, censorship, and government access. These constraints are beyond the company’s control and should not be interpreted as a reflection of DeepSeek’s technical capabilities. However, certain deficiencies (e.g. security and safety) appear to result from the lack of focus, likely due to minimal domestic pressure to prioritize these aspects. As such, while some limitations are systemic, others reflect the company’s priorities. Regardless, these are the “Bad” things you must consider seriously before deciding how you will use it.

1. Bias and Censorship

Starting with the obvious, Chinese AI labs must comply with stringent state-imposed censorship. Therefore, DeepSeek is trained to align with Chinese government narratives, and it will reliably censor certain information and avoid sensitive topics. This leads to highly biased responses. When prompted about contentious issues regarding China, such as the Tiananmen Square incident or the situation of Uyghurs in Xinjiang, DeepSeek will only respond in alignment with China’s position while ignoring all contrary evidence.

DeepSeek will even reinterpret historical accounts and give highly biased (sometimes completely untrue) answers that reflect China’s view of the world. For example, DeepSeek has been reported to misrepresent former U.S. President Jimmy Carter’s stance on Taiwan by falsely suggesting that he endorsed China’s claim over the island. Such biased behavior extends far beyond history and is observed even for recent events. When asked about China’s handling of the COVID-19 pandemic or Russia’s invasion of Ukraine, the New York Times found DeepSeek’s responses to be heavily biased, consistently favoring China’s geopolitical positions.

This selective presentation of information creates an uneven perception of events, especially among users not familiar with the background issues. It fosters distrust in professional journalism and undermines the foundation of democracy. Hence, NewsGuard researchers have referred to DeepSeek as a “disinformation machine.”

2. Privacy Challenges

Beyond censorship, DeepSeek must also comply with China’s strict data monitoring directives, which require companies to collect, store, and share data with state authorities upon request. According to DeepSeek’s privacy policy, it collects an extensive amount of personal data (e.g. emails, phone numbers, chat histories, search queries, device identifier, keystroke patterns, IP addresses, and internet activity from other apps, etc.). Furthermore, NowSecure’s analysis of DeepSeek’s iOS app confirmed that it collects “an awful lot of data about the user’s device that is on the edge of advanced device fingerprinting.” Direct government access to such data is clearly a huge privacy concern. As it would allow the government to track and monitor users, collect intelligence, and even manipulate users through targeted disinformation campaigns.

Exacerbating these concerns is the fact that DeepSeek offered no transparency into its data collection and storage practices in response to inquiries from data regulators, such as the Italian Data Protection Authority (DPA). Moreover, Feroot Security discovered obfuscated code within DeepSeek’s website capable of sending user data to China Mobile, a state-owned telecom.

It should be apparent that the Chinese regulatory environment is fundamentally incompatible with Western privacy standards. Consequently, governments around the world have either banned or restricted DeepSeek usage. Italy, Taiwan, South Korea, Australia, and multiple U.S. agencies (e.g. NASA, U.S. Navy, etc.) have blocked its use over national security and privacy risks. Even Texas has launched a formal investigation into DeepSeek’s compliance with state law. Without comprehensive data protection laws, DeepSeek users are exposed to increased risk of unauthorized data access, misuse, and potential exploitation.

3. Security Vulnerabilities

Numerous investigations revealed severe security vulnerabilities in DeepSeek. Cybersecurity firms have uncovered that DeepSeek employs weak encryption methods, making it susceptible to SQL injection attacks. Additionally, the application has been found to transmit sensitive user and device data unencrypted over the internet, increasing the risk of data interception. Notably, the DeepSeek iOS app disables App Transport Security (ATS), an iOS platform-level protection designed to prevent sensitive data from being sent over unencrypted channels.

A significant security lapse was identified when Wiz Research uncovered a publicly accessible DeepSeek database. This database contained over a million lines of sensitive data, including chat history, API secrets, backend data, log streams, and operational details. Moreover, this database was not just accessible and readable, but also controllable within the DeepSeek environment. This basically allows anyone who finds it to run queries against the database to retrieve sensitive logs, actual plaintext chat messages, and even steal plaintext passwords and local files.

Finally, DeepSeek’s models, particularly R1, are notably vulnerable to prompt injection attacks. One evaluation reported that DeepSeek’s R1 model failed to detect or block any of the 50 malicious prompts designed to elicit toxic content, resulting in a shocking 100% attack success rate. And this brings us to the next deficiency of DeepSeek’s LLM: safety!

4. Safety Concerns

The safety challenges of DeepSeek are a direct consequence of its lax security measures. DeepSeek can be jailbroken easily via prompt injection and is susceptible to various jailbreak techniques like Crescendo, Bad Likert Judge, Deceptive Delight, Do Anything Now (DAN), and EvilBOT. Research from Tenable shows that DeepSeek can be prompted to develop malware, such as keyloggers and ransomware. These findings were further replicated by Kela, a cyberthreat intelligence organization, whose team successfully bypassed the model’s safety measures and ethical guidelines. This allowed them to elicit many malicious outputs from R1 (e.g. ransomware source code, fabricated sensitive content, detailed instructions for creating toxins and explosive devices, etc.).

IMHO, open-sourcing a powerful AI model with weak security and limited safety guardrails is not only dangerous but also irresponsible. Having access to the source code not only allows for the modification of the model’s functions but also exposes its internal safety mechanisms, making it easier to bypass them. Hence, the Center for Strategic and International Studies (CSIS) noted “DeepSeek’s lack of safety guardrails and open-source design allows malicious actors to perform actions that Western models prevent.”

Conclusion

While DeepSeek’s innovations in developing hyper-efficient LLMs are truly noteworthy, we must not overlook its inadequacies. As in the previous installment, we highlighted 4 deficiencies of DeepSeek’s LLMs (2 are systemic, and 2 reflect a lack of company prioritization).

1. Bias: Censorship-driven biases distort realities and perpetuate misinformation
2. Privacy: Extensive data collection under foreign jurisdiction risks exploitation
3. Security: Inadequate security compromises user data and system integrity
4. Safety: Insufficient safeguards facilitate the dissemination of harmful content

While security vulnerabilities can be patched and safety guardrails can be implemented, both will require DeepSeek to make them a priority and devote time and resources to address these challenges. However, it’s unlikely that DeepSeek can or will address the bias and privacy issues.

Now that we’ve learned the “Good” (i.e. the innovations) and the “Bad” (i.e. the inadequacies) of DeepSeek, stay tuned for the next installment, where we’ll discuss the “Ugly” side of this story.