Benchmarking Performa dan Efisiensi Respon Model AI ChatGPT dan DeepSeek Menggunakan Metode Eksperimen Komparatif
DOI:
https://doi.org/10.47065/bulletincsr.v6i6.1399Keywords:
Benchmarking; AI Performance; Response Efficiency; ChatGPT; DeepSeek; Comparative ExperimentAbstract
The rapid development of Large Language Models (LLMs) has led to the emergence of various artificial intelligence platforms, such as ChatGPT and DeepSeek, which are relied upon for data processing efficiency. However, architectural differences between platforms raise issues regarding variations in response efficiency and information reliability when handling user queries. This study aims to conduct a benchmarking analysis to objectively measure the performance and response efficiency of ChatGPT and DeepSeek. The implemented solution employs a comparative experimental method through the analysis of 10,000 interaction records from a synthetic dataset available on Kaggle. Three primary evaluation metrics were measured: response speed (in seconds), response accuracy (in percentage), and user satisfaction (User Rating on a 1–5 scale). The benchmarking analysis results indicate that DeepSeek performs better than ChatGPT across the three evaluated indicators. DeepSeek processes responses with an average speed of 1.24 seconds, significantly faster than ChatGPT, which requires 3.44 seconds. In terms of accuracy, DeepSeek achieves an accuracy rate of 89.97%, whereas ChatGPT reaches 80.26%. This is consistent with user satisfaction, where DeepSeek earns a User Rating of 4.80 out of 5.00 compared to ChatGPT's 4.00. This study makes two contributions. First, it provides empirical benchmarks of the comparative performance of both AI models on three indicators within a single dataset, to support the selection of an optimal language processing system. Second, it provides comparison calculations with formulas and calculation steps, so that the results can be traced and re-checked. The results apply to the analyzed data.
Downloads
References
A. J. Thirunavukarasu, D. S. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. Ting, “Large language models in medicine,” Nat. Med., vol. 29, no. 8, pp. 1930–1940, 2023, doi: 10.1038/s41591-023-02448-8.
OpenAI, “GPT-4 technical report,” arXiv:2303.08774, 2023, doi: 10.48550/arXiv.2303.08774.
DeepSeek-AI, “DeepSeek-V3 technical report,” arXiv:2412.19437, 2024, doi: 10.48550/arXiv.2412.19437.
DeepSeek-AI, “DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning,” Nature, vol. 645, pp. 633–638, 2025, doi: 10.1038/s41586-025-09422-z.
M. Poo, “Reflections on DeepSeek’s breakthrough,” Natl. Sci. Rev., vol. 12, no. 3, art. no. nwaf044, 2025, doi: 10.1093/nsr/nwaf044.
S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal, “Detecting hallucinations in large language models using semantic entropy,” Nature, vol. 630, no. 8017, pp. 625–630, 2024, doi: 10.1038/s41586-024-07421-0.
Y. Chang et al., “A survey on evaluation of large language models,” ACM Trans. Intell. Syst. Technol., vol. 15, no. 3, pp. 1–45, 2024, doi: 10.1145/3641289.
A. Liesenfeld and M. Dingemanse, “Rethinking open source generative AI: Open-washing and the EU AI Act,” in Proc. ACM Conf. Fairness, Accountability, and Transparency (FAccT ’24), Rio de Janeiro, Brazil, 2024, pp. 1774–1784, doi: 10.1145/3630106.3659005.
A. Meyer, Y. Karay, A. U. Steinbicker, T. Streichert, and R. Overbeek, “Performance of DeepSeek-R1, ChatGPT (GPT-o3-mini), and Gemini 2.0 Flash on German medical multiple-choice questions: Comparative evaluation,” JMIR Form. Res., vol. 9, art. no. e77357, 2025, doi: 10.2196/77357.
W. Etaiwi and B. Alhijawi, “Comparative evaluation of ChatGPT and DeepSeek across key NLP tasks: Strengths, weaknesses, and domain-specific performance,” arXiv:2506.18501, 2025, doi: 10.48550/arXiv.2506.18501.
Q. Jiang, Z. Gao, and G. E. Karniadakis, “DeepSeek vs. ChatGPT vs. Claude: A comparative study for scientific computing and scientific machine learning tasks,” arXiv:2502.17764, 2025, doi: 10.48550/arXiv.2502.17764.
S. Singh, S. Bansal, A. El Saddik, and M. Saini, “From ChatGPT to DeepSeek AI: A comprehensive analysis of evolution, deviation, and future implications in AI-language models,” arXiv:2504.03219, 2025, doi: 10.48550/arXiv.2504.03219.
A. Rahman et al., “Comparative analysis based on DeepSeek, ChatGPT, and Google Gemini: Features, techniques, performance, future prospects,” arXiv:2503.04783, 2025, doi: 10.48550/arXiv.2503.04783.
Y. Peng et al., “From GPT to DeepSeek: Significant gaps remain in realizing AI in healthcare,” J. Biomed. Inform., vol. 163, art. no. 104791, 2025, doi: 10.1016/j.jbi.2025.104791.
A. Temsah et al., “DeepSeek in healthcare: Revealing opportunities and steering challenges of a new open-source artificial intelligence frontier,” Cureus, vol. 17, no. 2, art. no. e79221, 2025, doi: 10.7759/cureus.79221.
M. E. Kayaalp et al., “DeepSeek versus ChatGPT: Multimodal artificial intelligence revolutionizing scientific discovery. From language editing to autonomous content generation—redefining innovation in research and practice,” Knee Surg. Sports Traumatol. Arthrosc., vol. 33, no. 5, pp. 1553–1556, 2025, doi: 10.1002/ksa.12628.
P. Xu et al., “DeepSeek-R1 outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in bilingual complex ophthalmology reasoning,” Adv. Ophthalmol. Pract. Res., vol. 5, no. 3, pp. 189–195, 2025, doi: 10.1016/j.aopr.2025.05.001.
D. Mikhail et al., “Performance of DeepSeek-R1 in ophthalmology: An evaluation of clinical decision-making and cost-effectiveness,” Br. J. Ophthalmol., vol. 109, no. 9, pp. 976–981, 2025, doi: 10.1136/bjo-2025-327360.
A. Arrieta, M. Ugarte, P. Valle, J. A. Parejo, and S. Segura, “o3-mini vs DeepSeek-R1: Which one is safer?,” arXiv:2501.18438, 2025, doi: 10.48550/arXiv.2501.18438.
M. Liu et al., “Performance of ChatGPT across different versions in medical licensing examinations worldwide: Systematic review and meta-analysis,” J. Med. Internet Res., vol. 26, art. no. e60807, 2024, doi: 10.2196/60807.
M. Alfertshofer et al., “Analyzing question characteristics influencing ChatGPT’s performance in 3000 USMLE-style questions,” Med. Sci. Educ., vol. 35, no. 1, pp. 257–267, 2025, doi: 10.1007/s40670-024-02176-9.
A. Meyer, J. Riese, and T. Streichert, “Comparison of the performance of GPT-3.5 and GPT-4 with that of medical students on the written German medical licensing examination: Observational study,” JMIR Med. Educ., vol. 10, art. no. e50965, 2024, doi: 10.2196/50965.
A. Khan, “DeepSeek vs ChatGPT: AI Platform Comparison,” Kaggle, dataset versi 8. [Online]. Available: https://www.kaggle.com/datasets/khanaakif/deepseek-vs-chatgpt-ai-platform-comparison. Accessed: Oct. 16, 2026.
W.-L. Chiang et al., “Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference”, ICML, vol. 235, pp. 8359-8388, 2024. https://proceedings.mlr.press/v235/chiang24b.html
L. Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”, NeurIPS, vol. 36, 2023. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract.html
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Benchmarking Performa dan Efisiensi Respon Model AI ChatGPT dan DeepSeek Menggunakan Metode Eksperimen Komparatif
ARTICLE HISTORY
How to Cite
Issue
Section
Copyright (c) 2026 Siti Intan Nurfadilah, Al Imran, Sandi Firmansyah, Muhammad Alamsyah, Farizi Ilham
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
