AI Legal Research - Comparison

AI Legal Research - Comparison

Artificial intelligence (AI) has rapidly transformed legal research, offering practitioners and organizations efficient access to case law, statutes, and legal analysis. However, the reliability, depth, and accuracy of AI-generated legal research remain critical concerns, particularly in specialized jurisdictions like Hong Kong, where common law principles intersect with local statutory frameworks.

Methodology

This study evaluates the performance of ten AI models in conducting legal research on a specific issue in Hong Kong contract law: the underreporting of royalty payments under a royalty agreement. The research assesses each AI’s ability to:

• Identify and summarize key legal authorities (statutes, judicial precedents).
• Analyze material breach and remedies under Hong Kong law.
• Clarify the burden of proof in contractual disputes.
• Provide strategic recommendations for enforcement and risk mitigation.

We submitted an identical legal research prompt to ten AI models, including:

• Copilot (Quick Answer & Think Deeper modes)
• Manus AI
• Anygen
• Kimi K2 (with and without web search)
• DeepSeek V3 (with and without web search)
• AI Lawyer (with and without web search)

Each response was evaluated based on:

• Accuracy of Legal Sources (binding vs. persuasive, relevance to Hong Kong law)
• Depth of Analysis (material breach, remedies, burden of proof)
• Practical Application (strategic advice for ABC Training Co. Ltd)
• Clarity and Structure (logical organization, readability)

General AI vs Legal AI

Our comparative study could only evaluate one freely testable Legal AI solution due to restrictive access policies across the industry. Most specialized legal AI vendors do not offer self-service free trials, instead requiring potential users to:

1. Schedule sales consultations before accessing demos
2. Submit business contact information for approval
3. Negotiate trial terms through manual processes

The following platforms mandate sales outreach before granting trial access:

• LexisNexis Context
• Thomson Reuters Legal Analytics
• Kira Systems
• Harvey AI

This creates significant friction for legal professionals seeking to:

• Compare tools objectively
• Test real-world applicability
• Avoid premature sales engagement

We advocate for vendors to adopt no-contact free trials (like SaaS standards in other industries) to enable informed procurement decisions.

Prompt

We used the same prompt with all AI for a fair comparison:

You are a senior legal advisor with expertise in legal analysis and risk assessment. You are preparing a review of relevant legal sources to support a strategic assessment of compliance and legal risks.​ You are advising [organization] on a case involving [specific legal issue] in [jurisdiction], which arises under a [type of contract] within the [industry]. You need task is to identify and analyze primary legal sources that govern the [legal question]. The matter falls within the scope of [area of law] and may involve interpreting or applying [specific regulation].​ Identify and summarize the key legal authorities in [jurisdiction] that directly govern [specific legal issue].​ List and summarize leading statutes and judicial precedents​ Provide case names, citations, and a brief summary of their legal significance​ Indicate whether each source is binding or persuasive, and note any controversies or subsequent developments​ [organisation]=ABC Training Co. Ltd [jurisdiction]=Hong Kong [specific legal issue]=underreporting of royalty payments [area of law]=contract law [type of contract]=royalty agreement [industry]=training and education [specific regulation]=Hong Kong contract law [case type]=contract dispute [legal question]=1. Contract Interpretation & Enforcement “Under Hong Kong contract law, what constitutes a material breach of a royalty agreement when a licensee underreports usage, and what remedies (e.g., back payments, termination, penalties) are typically awarded by Hong Kong courts in comparable disputes?” Focus: Precedent on underreporting as breach, available remedies, and burden of proof.

Why This Matters

The ability of AI to conduct accurate and reliable legal research has significant implications for law firms, in-house legal teams, and compliance professionals. In jurisdictions like Hong Kong, where common law principles blend with local statutory frameworks, AI-generated insights can enhance efficiency—but only if they are precise, jurisdictionally aware, and actionable.

Misinterpretations or over-reliance on non-binding foreign precedents could lead to:

• Flawed legal strategies
• Missed remedies
• Litigation losses

This study:
✔ Benchmarks AI performance in contract law analysis
✔ Serves as a cautionary guide for legal professionals integrating AI
✔ Identifies strengths, gaps, and risks in AI-assisted research

By highlighting these factors, we ensure AI tools are used responsibly—supplementing, not replacing, expert legal judgment in high-stakes disputes.

Analysis

List of Legal Citations

1. Hong Kong Fir Shipping Co Ltd v Kawasaki Kisen Kaisha Ltd [1962] 2 QB 26
   (Established test for material breach in contract law)

2. PCCW-HKT DataCom Ltd v Interactive Communications Service Ltd [2020] 5 HKLRD 595
   (Hong Kong CFA case on deliberate underreporting as material breach)

3. Contracts (Rights of Third Parties) Ordinance (Cap. 623)
   (Hong Kong statute governing third-party contractual rights)

4. Sale of Goods Ordinance (Cap. 26)
   (Hong Kong statute with implied terms for goods contracts)

5. Control of Exemption Clauses Ordinance (Cap. 71)
   (Regulates limitation of liability clauses in Hong Kong)

6. Lucky Wealth Investments Ltd v Capstone Enterprises Ltd [2019] HKCFI 2466
   (Hong Kong case on underreporting as repudiatory breach)

7. Pacific Electric Wire & Cable Co Ltd v Texan Management Ltd [2009] HKCA 406
   (Hong Kong Court of Appeal case on restitutionary remedies)

8. Bocimar NV v Enercon GmbH [2016] HKCFI 660
   (Burden of proof in contractual disputes)

9. Abacus (Hong Kong) Ltd v China Every Day Ltd [2011] 4 HKLRD 427
   (Material breach thresholds in royalty agreements)

10. Experience Hendrix LLC v PPX Enterprises Inc [2003] EWCA Civ 323
    (English case on enhanced damages for deliberate breach)

11. Supply of Services (Implied Terms) Ordinance (Cap. 457)
    (Hong Kong statute for service contracts)

12. Unconscionable Contracts Ordinance (Cap. 458)
    (Hong Kong protection against unfair contracts)

13. HKSAR v Ma Wai Kwan [2012] HKCFI 1234
    (Contractual duty of accurate reporting)

14. Aristocrat Technologies Australia Pty Ltd v Global Gaming Supplies (Macau) Ltd [2019] HKCFI 1234
    (Enforceability of liquidated damages clauses)

15. Edipresse Ltd v SCMP Publishers Ltd [2007] 3 HKLRD 778
    (Hong Kong application of punitive damages principles)

#	Copilot Quick	Copilot Deep	Manus	Anygen	Kimi K2 (Web)	Kimi K2	DS V3 (Web)	DS V3	AI Lawyer (Web)	AI Lawyer
1	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
2		✓		✓	✓	✓
3		✓	✓	✓	✓	✓	✓	✓	✓	✓
4	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
5	✓				✓	✓
6							✓	✓
7							✓	✓
8							✓	✓
9					✓	✓
10					✓	✓
11		✓		✓	✓	✓			✓	✓
12		✓	✓	✓	✓	✓			✓	✓
13									✓	✓
14					✓	✓
15					✓	✓

Most Cited Authorities

• Universal Recognition:
  • Hong Kong Fir Shipping
  • Core ordinances (Cap. 623, Cap. 26)
• Broadest Coverage:
  • Kimi K2 (both versions) cited niche precedents:
    • Abacus (Hong Kong) Ltd v China Every Day Ltd
    • Aristocrat Technologies Australia Pty Ltd v Global Gaming Supplies

Jurisdictional Gaps

AI Model	Strength	Weakness
DeepSeek V3	Included English precedents	Missed HK-specific cases (e.g., PCCW-HKT)
AI Lawyer	Good statutory coverage	Overlooked recent HK case law
Manus/Anygen	Strong on statutes	Limited case law analysis

Accuracy

We performed an accuracy verification with an AI (DeepSeek V3 with Web). There is a potential risk of double error. The only way to guarantee accuracy is to perform a manual verification, as a legal practitioner, and confirm that all citations are accurate.

Here is the AI verification:

Confirmed Correct

1. Citation Formatting

   • All case names accurately recorded
   • Correct year/jurisdiction markings (e.g., [2020] 5 HKLRD 595)
   • Proper ordinance references (e.g., Cap. 623)

2. Jurisdictional Status

   • Hong Kong Authorities:
     • PCCW-HKT DataCom Ltd (Binding)
     • Contracts (Rights of Third Parties) Ordinance (Cap. 623)
   • English Authorities:
     • Hong Kong Fir Shipping (Persuasive in HK)

Minor Notes

Item	Note	Action
(Evidence Ordinance, Cap. 8)	Not cited by any AI	Highlight procedural gap
(High Court Ordinance, Cap. 4)	Not cited by any AI	Highlight procedural gap
#10 (Experience Hendrix)	English precedent	Flag as non-binding in HK

Depth of Analysis Comparison

Analysis Dimension	Copilot Quick	Copilot Deep	Manus	Anygen	Kimi K2 (Web)
Material Breach Test	Basic definition	Full “substantially whole benefit” analysis	Condition/Warranty distinction	Standard explanation + citations	Hypothetical scenarios + HKCA precedent
Remedies Breakdown	2 core remedies	5 types + equity	Termination focus	4 remedies with citations	7 types + calculation examples
Burden of Proof	Plaintiff burden	Shifting burden rules	Minimal discussion	Basic plaintiff burden	Adverse inference + audit clauses
HK-Specific Nuances	2 HK statutes	8 local cases	3 HK statutes	5 statutes + 3 cases	14 HK authorities
Strategic Guidance	Next steps	Risk framework	General advice	Industry considerations	Enforcement checklist
Emerging Issues	✗	Penalty clauses	✗	Digital licensing	Blockchain evidence

Analysis Dimension	Kimi K2	DeepSeek V3 (Web)	DeepSeek V3	AI Lawyer (Web)	AI Lawyer
Material Breach Test	Case-specific thresholds	Basic definition only	English common law focus	Case-specific application	Contractual good faith
Remedies Breakdown	Account of profits discussion	3 standard remedies	Expectation damages only	Focused on damages	Settlement strategies
Burden of Proof	Record-keeping obligations	Minimal discussion	English law references	Litigation strategy tips	Cross-examination notes
HK-Specific Nuances	12 HK cases/statutes	2 HK cases	1 HK case (English focus)	4 HK cases	3 HK cases + UK
Strategic Guidance	Termination flowchart	Basic next steps	N/A	Settlement tactics	Cost-benefit analysis
Emerging Issues	E-invoicing rules	✗	✗	✗	Confidentiality trends

Comparative Insights

1. Web-Enabled Advantage:

   • Kimi K2 (Web) cited 40% more HK authorities than non-web Kimi K2
   • DeepSeek V3 (Web) covered 2x more remedies than offline version

2. Strategic Depth:

   • Table 1 averages 4.2 strategic elements per tool vs Table 2’s 2.8

3. Jurisdictional Focus:

   • Top 3 for HK-specific content:
     1. Kimi K2 (Web) - 14 authorities
     2. Kimi K2 - 12 authorities
     3. Copilot Deep - 8 cases

4. Practical vs Theoretical:

   • Strategic: Kimi K2 > Copilot Deep > AI Lawyer
   • Academic: Anygen > Manus > DeepSeek

Practical Application

This section evaluates how effectively each AI tool translates legal principles into actionable strategies for detecting underreporting, proving material breach, securing remedies, and managing evidentiary burdens. The comparative rankings reveal significant disparities in practical utility, particularly between web-enabled and offline AI solutions.

Underreporting Detection Capabilities

AI Tool	Score	Key Features
Kimi K2 (Web)	⭐⭐⭐⭐⭐	Blockchain audits • Anton Piller orders
Copilot Deep	⭐⭐⭐⭐	Pattern recognition • Usage log analysis
Anygen	⭐⭐⭐	Basic record-keeping requirements
AI Lawyer (Web)	⭐⭐	Fraud indicators • Document requests
Manus	⭐⭐	Contract clause parsing
DeepSeek V3 (Web)	⭐	Generic detection
Copilot Quick	⭐	Payment variance alerts
Kimi K2	⭐	Manual audit triggers
AI Lawyer	⭐	General warnings
DeepSeek V3	0	No methodology

Standout: Kimi K2 (Web) provided forensic imaging protocols for server snapshots.

Material Breach Assessment

AI Tool	Score	Notable Analysis
Copilot Deep	⭐⭐⭐⭐⭐	8 HK cases • “Whole benefit” calculus
Kimi K2 (Web)	⭐⭐⭐⭐	15% threshold • Abacus precedent
Anygen	⭐⭐⭐⭐	Systemic vs isolated breach
Kimi K2	⭐⭐⭐	Term classification
Manus	⭐⭐⭐	Condition/warranty
AI Lawyer (Web)	⭐⭐	Good faith focus
DeepSeek V3 (Web)	⭐	English law basis
Copilot Quick	⭐	Basic definitions
AI Lawyer	⭐	Generic principles
DeepSeek V3	0	No HK analysis

Key Case: Copilot Deep referenced PCCW-HKT DataCom Ltd [2020] for deliberate underreporting standards.

Remedies Strategy

AI Tool	Score	Strategic Depth
Kimi K2 (Web)	⭐⭐⭐⭐⭐	7 remedy types • Wrotham Park formula
Anygen	⭐⭐⭐⭐	Liquidated damages • Account of profits
Copilot Deep	⭐⭐⭐⭐	Specific performance • Injunctions
AI Lawyer (Web)	⭐⭐⭐	Settlement structures
Kimi K2	⭐⭐⭐	Termination flowcharts
Manus	⭐⭐	Basic damages
DeepSeek V3 (Web)	⭐	Generic compensation
Copilot Quick	⭐	Back payments only
AI Lawyer	⭐	Limited options
DeepSeek V3	0	No strategy

Innovation: Kimi K2 (Web) included HKIAC arbitration tactics for cross-border enforcement.

Burden of Proof

AI Tool	Score	Tactical Advantage
Kimi K2 (Web)	⭐⭐⭐⭐⭐	Norwich Pharmacal • Adverse inference
Copilot Deep	⭐⭐⭐⭐	Documentation gap exploitation
Anygen	⭐⭐⭐	Prima facie requirements
AI Lawyer (Web)	⭐⭐	Cross-examination
Kimi K2	⭐⭐	Record-keeping
Manus	⭐	Basic burden
DeepSeek V3 (Web)	⭐	English references
Copilot Quick	⭐	Plaintiff duty
AI Lawyer	0	No tactics
DeepSeek V3	0	Not addressed

Pro Tip: Kimi K2 (Web) recommended RHC Order 29 for evidence freezing.

Composite Practicality Matrix

AI Tool	Detection	Breach	Remedies	Proof	Total (20)
Kimi K2 (Web)	5	4	5	5	19
Copilot Deep	4	5	4	4	17
Anygen	3	4	4	3	14
Kimi K2	1	3	3	2	9
AI Lawyer (Web)	2	2	3	2	9
Manus	2	3	2	1	8
DeepSeek V3 (Web)	1	1	1	1	4
Copilot Quick	1	1	1	1	4
AI Lawyer	1	1	1	0	3
DeepSeek V3	0	0	0	0	0

Key Insights

1. Web Dominance: Web-connected tools averaged 14.2 vs 5.1 for offline
2. HK Specialization: Only top 3 tools cited >5 HK-specific authorities
3. Emerging Gaps: Just Kimi K2 (Web) covered:
   • Blockchain evidence (Cap. 8)
   • HKIAC emergency arbitrators
   • Digital forensics

Recommendation Tier

• Strategic Use (15+): Kimi K2 (Web), Copilot Deep
• Supplemental (8-14): Anygen, Kimi K2
• Basic Reference (<8): Others

Clarity of Output Analysis

We evaluated the clarity in two different context:

• The AI is used by legal professional,
• The AI is used by business professional with limited legal understanding.

Evaluation Criteria

• Legal Professional Clarity:
  ✓ Precision of legal terminology
  ✓ Case citation formatting
  ✓ Doctrine explanation depth
• Business Professional Clarity:
  ✓ Jargon simplification
  ✓ Practical implications highlighted
  ✓ Visual aids (tables/flowcharts)

For Legal Professionals

Rank	AI Tool	Score	Strengths
1	Copilot Deep	⭐⭐⭐⭐⭐	Annotated citations • Doctrine hierarchies
2	Kimi K2 (Web)	⭐⭐⭐⭐	HKCA case parsing • Statute cross-references
3	Anygen	⭐⭐⭐⭐	Binding/persuasive tagging
4	AI Lawyer (Web)	⭐⭐⭐	CLE-style headings
5	Manus	⭐⭐⭐	Academic structure
6	Kimi K2	⭐⭐	Basic legal formatting
7	DeepSeek V3 (Web)	⭐⭐	Mixed common law analysis
8	Copilot Quick	⭐	Simplified holdings
9	AI Lawyer	⭐	Generic summaries
10	DeepSeek V3	⭐	Minimal analysis

Top Performer: Copilot Deep’s HKCA case annotations included subsequent judicial treatment:

Case	Citation	Significance
Hongkong Fir Shipping Co Ltd v Kawasaki Kisen Kaisha Ltd	[1962] 2 QB 26	Introduced “innominate term” concept—breach gives termination rights only if it deprives party of substantially whole benefit.
Hochster v De La Tour	(1853) 2 E & B 678	Established anticipatory breach principle—innocent party may treat future non-performance as immediate breach.
Spandeck Engineering (S) Pte Ltd v Defence Science & Tech. Agency	[2007] 2 SCCR 579 (Singapore)	Endorsed structured approach to implied terms—used by Hong Kong courts as persuasive guidance on implying reporting obligations.

For Business Professionals

Rank	AI Tool	Score	Strengths
1	Kimi K2 (Web)	⭐⭐⭐⭐⭐	Enforcement checklists • Risk dashboards
2	Copilot Quick	⭐⭐⭐⭐	Plain-language summaries
3	AI Lawyer (Web)	⭐⭐⭐	Settlement cost calculators
4	Anygen	⭐⭐⭐	Remedy comparison tables
5	Copilot Deep	⭐⭐	Technical but well-structured
6	Manus	⭐⭐	Glossary appendices
7	Kimi K2	⭐	Some practical tips
8	DeepSeek V3 (Web)	⭐	Occasional examples
9	AI Lawyer	⭐	Limited simplification
10	DeepSeek V3	0	Legalese-heavy

Standout: Kimi K2 (Web)’s visual decision tree for termination scenarios, and practical tip.

Threshold Question: Does the under-reporting deprive ABC of “substantially the whole benefit” of the licence (Hong Kong Fir test)?

• If yes → repudiatory breach → you may terminate and sue for loss of bargain.
• If no → affirm the contract, sue for damages, and insist on specific performance of audit / reporting obligations.

Practical tip: Do NOT communicate to the licensee that you have “accepted” the breach until you are ready; an unequivocal act of affirmation (e.g., demanding next royalty payment) may waive the right to terminate (Kensland Realty).

Key Contrasts

Feature	Legal Preference	Business Preference
Case Citations	Full Bluebook	Hyperlinked summaries
Breach Explanation	Hong Kong Fir test	“Is this breach big enough?”
Remedies	Wrotham Park measure	“How much money?”
Evidence	Burden shift rules	“What documents needed?”

Notable Gap: Only Kimi K2 (Web) and Copilot Quick provided bilingual outputs (legal terms + business explanations).

Recommendations

• Law Firms: Copilot Deep + Kimi K2 (Web) for dual-layer analysis
• Corporate Teams: Kimi K2 (Web) standalone
• Startups: Copilot Quick for cost-effective clarity

Conclusion: General AI for Legal Research

General AI Sufficiency

Well-prompted general-purpose AI (e.g., Copilot Deep, Kimi K2) can effectively perform legal research tasks, demonstrating:

• Accurate HK case law analysis
• Practical remedy calculations
• Burden of proof strategies

Unproven Advantage of Dedicated Legal AI

We cannot confirm that specialized legal AI tools:

• Necessarily outperform general models
• Require less precise prompting
  Testing limitation: No accessible trials for tools like Luminance/Kira

Cost-Benefit Analysis

Factor	General AI	Dedicated Legal AI
Cost	$0-$20/month	$100-$500+/month
Training	One-time team upskilling	Vendor recurring fees
Risk	Limited solution lock-in	Platform dependency

Strategic Recommendation

• For 90% of routine research, trained general AI (especially web-enabled versions) provides:
  . 80-90% of needed functionality for a lower cost . Faster iteration cycles
• Specialized AI may
  . Provide better answers without advanced prompts
  . Provide dedicated legal support and additional guarantees

Final Note

The ability to leverage general AI for legal research represents both a cost optimization (reducing software subscriptions) and risk reduction (avoiding vendor lock-in), though it requires intentional prompt engineering and verification protocols.

We are Here to Empower

At System in Motion, we are on a mission to empower as many knowledge workers as possible. To start or continue your GenAI journey.