Ethics and Governance of General Models: Challenges and Countermeasures

Authors

Yan TENG, Governance Research Center, Shanghai Artificial Intelligence Laboratory, Shanghai 200232, ChinaFollow
Guoyu WANG, School of Philosophy, Fudan University, Shanghai 200433, China; Center for Biomedical Ethics, Fudan University, Shanghai 200433, ChinaFollow
Yingchun WANG, Governance Research Center, Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China

Keywords

artificial intelligence; general model; risk; governance; ethics of technology

Document Type

Policy & Management Research

Abstract

In recent years, the general model is one of the most important development trends of artificial intelligence. With the rapidly increasing research and deployment of general models, the social and ethical effects of general models have received extensive attention. Grounded in the characteristics of general models, this article analyzes the potential ethical challenges of the models at three levels:algorithm, data, and computing power. The detailed challenges include uncertainty, truthfulness, reliability, bias, toxicity, fairness, privacy, and environmental issues. Also, through the lens of philosophy of technology, it elaborates the important reasons for the ethical challenges:the "mirroring" effect and transparency problem caused by the data-driven general models' mediation between human and the world. This relation can be depicted as "human-model (data)-world". Finally, from the perspectives of governance tools and governance mechanisms, this article reviews the current countermeasures and reflects on their limitations. It is recommended to establish an open, full-process, value-embedded ethical restraint mechanism to ensure that the general model develops in accordance with legal and ethical requirements.

First page

1290

Last Page

1299

Language

Chinese

Publisher

Bulletin of Chinese Academy of Sciences

References

1 Knight J. Computer Modeling. (2017-06-11)[2022-03-06]. https://www.encyclopedia.com/social-sciences-and-law/law/crime-and-law-enforcement/computer-modeling.

2 Bommasani R, Hudson D A, Adeli E, et al. On the Opportunities and Risks of Foundation Models. San Francisco:Stanford University, 2021.

3 黄铁军, 文继荣, 刘知远, 等. 超大规模智能模型产业发展报告. 北京:北京智源人工智能研究院, 2021.

Huang T J, Wen J L, Liu Z Y, et al. Super Large Scale Intelligent Model Industry Development Report. Beijing:Beijing Academy of Artificial Intelligence, 2021. (in Chinese)

4 徐英瑾, 陈萌. 人工智能如何"说人话"?——对于自然语言处理研究的哲学反思. 自然辩证法通讯, 2022, 44(1):10- 19.

Xu Y J, Chen M. How to make artificial intelligence capable of speaking human language? Some philosophical remarks on natural language processing. Journal of Dialectics of Nature, 2022, 44(1):10-19. (in Chinese)

5 Clark E, August T, Serrano S, et al. All that's 'human' is not gold:Evaluating human evaluation of generated text//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2021:7282-7296.

6 Shao J, Chen S, Li Y, et al. Intern:A new learning paradigm towards general vision. arXiv preprints, 2021:2111.08687.

7 Goldstein J. Emergence as a construct:History and issues. Emergence, 1999, 1(1):49-72.

8 赵斌. 充分理解涌现性, 慎重对待转基因. 科学家, 2013, 12(2):88-89.

Zhao B. Comprehending emergence to think differently about genetically modified technology. Scientist, 2013, 12(2):88-89. (in Chinese)

9 Samuel A L. Some studies in machine learning using the game of checkers. II-Recent progress. IBM Journal of Research and Development, 1967, 11(6):601-617.

10 Steinhardt J. On the risks of emergent behavior in foundation models. (2021-10-18)[2022-03-05]. https://crfm.stanford.edu/commentary/2021/10/18/steinhardt.html.

11 Hendrycks D, Carlini N, Schulman J, et al. Unsolved problems in ML safety. arXiv preprints, 2021:2109.13916.

12 Szegedy C, Zaremba W, Sutskever I, et al. Intriguing properties of neural networks. arXiv preprints, 2014:1312.6199.

13 Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXic preprints, 2015:1412.6572v3.

14 Goh G, Cammarata N, Voss C, et al. Multimodal neurons in artificial neural networks. (2021-03-04)[2022-03-05]. https://distill.pub/2021/multimodal-neurons/.

15 Schwartz R, Vassilev A, Greene K, et al. Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. Gaithersburg:National Institute of Standards and Technology, 2022.

16 Radford A, Kim J W, Hallacy C, et al. Learning transferable visual models from natural language supervision. arXiv preprints, 2021:2103.00020.

17 王国豫, 梅宏. 构建数字化世界的伦理秩序. 中国科学院院刊, 2021, 36(11):1278-1287.

Wang G Y, Mei H. Constructing ethical order of digital ?は?????あ??ぬ??ど??of Chinese Academy of Sciences, 2021, 36(11):1278- 1287. (in Chinese)

18 Weidinger L, Mellor J, Rauh M, et al. Ethical and social risks of harm from language models. arXiv preprints, 2021:2112.04359.

19 Sensity. The state of deepfakes:Landscape, threats, and impact. (2019-11-29)[2022-03-05]. https://medium.com/sensity/mapping-the-deepfake-landscape-27cb809e98bc

20 Strubell E, Ganesh A, McCallum A. Energy and policy considerations for deep learning in NLP//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence:Association for Computational Linguistics, 2019:3645-3650.

21 Patterson D, Gonzalez J, Le Q, et al. Carbon emissions and large neural network training. arXiv preprints, 2021:2104.10350.

22 Bender E M, Gebru T, Mcmillan-Major A, et al. On the dangers of stochastic parrots:Can language models be too big?//Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. New York:Association for Computing Machinery, 2021:610-623.

23 Hey T, Tansley S, Tolle K. Jim Gray on eScience:A transformed scientific method//The Fourth Paradigm:DataIntensive Scientific Discovery. Remond:Microsoft Research, 2009.

24 C.胡必希, 王国豫. 技术评估的方法与价值冲突. 自然辩证法研究, 2005, 21(12):40-43.

Christoph H B, Wang G Y. The method of technology assessment and value conflict. Studies in Dialectics of Nature, 2005, 21(12):40-43. (in Chinese)

25 Ihde D. Technology and the Lifeworld:From Garden to Earth. Bloomington:Indiana University Press, 1990.

26 Welbl J, Glaese A, Uesato J, et al. Challenges in detoxifying language models. arXiv preprints, 2021:2109.07445.

27 Solaiman I, Dennison C. Process for adapting language models to society (PALMS) with values-targeted datasets. arXiv preprints, 2021:2106.10328.

28 Thoppilan R, Freitas D D, Hall J, et al. LaMDA:Language models for dialog applications. arXiv preprints, 2022:2201.08239.

29 Solaiman I, Brundage M, Clark J, et al. Release strategies and the social impacts of language models. arXiv preprints, 2019:

1908.09203.

Recommended Citation

TENG, Yan; WANG, Guoyu; and WANG, Yingchun (2022) "Ethics and Governance of General Models: Challenges and Countermeasures," Bulletin of Chinese Academy of Sciences (Chinese Version): Vol. 37 : Iss. 9 , Article 19.
DOI: https://doi.org/10.16418/j.issn.1000-3045.20220505003
Available at: https://bulletinofcas.researchcommons.org/journal/vol37/iss9/19