50 Amendments of Eva KAILI related to 2021/0106(COD)
Amendment 130 #
Proposal for a regulation
Recital 1
Recital 1
(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of trustworthy artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety, the environment, and fundamental rights, and it ensures the free movement of AI- based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.
Amendment 135 #
Proposal for a regulation
Recital 2 a (new)
Recital 2 a (new)
(2a) The deployment of artificial intelligence applications across sectors will only accelerate in the years to come. The European Union should therefore consider, in separate legislation, the creation of an Artificial Intelligence Adjustment Fund, which could be beneficial for Member States to cover the accustoming of their labour markets to the new conditions arising from the rapid mass introduction of artificial intelligence systems that could affect specific job sectors.
Amendment 168 #
Proposal for a regulation
Recital 16
Recital 16
(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive, access brain or brain-generated data without consent, or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human- machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research.
Amendment 171 #
Proposal for a regulation
Recital 17
Recital 17
(17) AI systems providing social scoring of natural persons for general purpose by public authorities or on their behalf, assessing the risk of natural person for offending or reoffending, or categorising persons based on biometrics or biometrics-based data, may lead to discriminatory outcomes and the exclusion of certain groups. They may violate the right to dignity and non- discrimination and the values of equality and justice. Such AI systems evaluate or classify the trustworthiness of natural persons based on their social behaviour in multiple contexts or known or predicted personal or personality characteristics. The social score or risk assessment obtained from such AI systems may lead to the detrimental or unfavourable treatment of natural persons or whole groups thereof in social and legal contexts, which are unrelated to the context in which the data was originally generated or collected or to a detrimental treatment that is disproportionate or unjustified to the gravity of their social behaviour. Such AI systems should be therefore prohibited.
Amendment 176 #
Proposal for a regulation
Recital 18
Recital 18
(18) The use of AI systems for ‘real- time’ remotebiometric or biometrics-based identification of natural persons in publicly accessible spaces for the purpose of law enforcement is considered particularly intrusive in the rights and freedoms of the concerned persons, to the extent that it may affect the private life of a large part of the population, evoke a feeling of constant surveillance and indirectly dissuade the exercise of the freedom of assembly and other fundamental rights. In addition, the immediacy of the impact and the limited opportunities for further checks or corrections in relation to the use of such systems operating in ‘real-time’ carry heightened risks for the rights and freedoms of the persons that are concerned by law enforcement activities.
Amendment 177 #
Proposal for a regulation
Recital 19
Recital 19
(19) The use of those systems for the purpose of law enforcement should therefore be prohibited, except in three exhaustively listed and narrowly defined situations, where the use is strictly necessary to achieve a substantial public interest, the importance of which outweighs the risks. Those situations involve the search for potential victims of crime, including missing children; certain threats to the life or physical safety of natural persons or of a terrorist attack; and the detection, localisation, identification or prosecution of perpetrators or suspects of the criminal offences referred to in Council Framework Decision 2002/584/JHA38 if those criminal offences are punishable in the Member State concerned by a custodial sentence or a detention order for a maximum period of at least three years and as they are defined in the law of that Member State. Such threshold for the custodial sentence or detention order in accordance with national law contributes to ensure that the offence should be serious enough to potentially justify the use of ‘real-time’ remote biometric identification systems. Moreover, of the 32 criminal offences listed in the Council Framework Decision 2002/584/JHA, some are in practice likely to be more relevant than others, in that the recourse to ‘real-time’ remote biometric identification will foreseeably be necessary and proportionate to highly varying degrees for the practical pursuit of the detection, localisation, identification or prosecution of a perpetrator or suspect of the different criminal offences listed and having regard to the likely differences in the seriousness, probability and scale of the harm or possible negative consequences. _________________ 38 Council Framework Decision 2002/584/JHA of 13 June 2002 on the European arrest warrant and the surrender procedures between Member States (OJ L 190, 18.7.2002, p. 1)biometric or biometrics- based systems that can be used for monitoring large numbers of persons, be it in public or private spaces, should therefore be prohibited.
Amendment 181 #
Proposal for a regulation
Recital 20
Recital 20
Amendment 184 #
Proposal for a regulation
Recital 21
Recital 21
Amendment 186 #
Proposal for a regulation
Recital 22
Recital 22
Amendment 187 #
Proposal for a regulation
Recital 23
Recital 23
Amendment 189 #
Proposal for a regulation
Recital 24
Recital 24
(24) Any processing of biometric data and other personal data involved in the use of AI systems for biometric identification, other than in connection to the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement as regulated by this Regulation, including where those systems are used by competent authorities in publicly accessible spaces for other purposes than law enforcement, should continue to comply with all requirements resulting from Article 9(1) of Regulation (EU) 2016/679, Article 10(1) of Regulation (EU) 2018/1725 and Article 10 of Directive (EU) 2016/680, as applicable.
Amendment 234 #
Proposal for a regulation
Recital 70
Recital 70
(70) Certain AI systems intended to interact with natural persons or to generate content may pose specific risks of impersonation or deception irrespective of whether they qualify as high-risk or not. In certain circumstances, the use of these systems should therefore be subject to specific transparency obligations without prejudice to the requirements and obligations for high-risk AI systems. In particular, natural persons should be notified that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. Moreover, natural persons should be notified when they are exposed to an emotion recognition system or a biometric categorisation system. Such information and notifications should be provided in accessible formats for persons with disabilities. Further, users, who use an AI system to generate or manipulate image, audio or video content that appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic, should disclose that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin.
Amendment 249 #
Proposal for a regulation
Recital 76
Recital 76
(76) In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence. The Board should work towards establishing a European Regulatory Agency for Artificial Intelligence in line with the provisions of Article 56(3).
Amendment 265 #
Proposal for a regulation
Article 2 – paragraph 5 a (new)
Article 2 – paragraph 5 a (new)
5a. This Regulation shall facilitate the exchange of data used solely for academic and scientific endeavours in a safe scientific space.
Amendment 273 #
Proposal for a regulation
Article 3 – paragraph 1 – point 1
Article 3 – paragraph 1 – point 1
(1) ‘artificial intelligence system’ (AI system) means any software or machine- based system that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content,make predictions, recommendations, or decisions influencing real or virtual environments. the environments they interact with;. AI systems can be designed with varying levels of autonomy.
Amendment 293 #
Proposal for a regulation
Article 3 – paragraph 1 – point 33 a (new)
Article 3 – paragraph 1 – point 33 a (new)
(33a) ‘biometrics-based data’ means personal data resulting from specific technical processing related to physical, physiological or behavioural signals or characteristics of a natural person, such as facial expressions, movements, pulse frequency, voice, keystrokes or gait, which may or may not allow or confirm the unique identification of a natural person;
Amendment 294 #
Proposal for a regulation
Article 3 – paragraph 1 – point 34
Article 3 – paragraph 1 – point 34
(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intention, thoughts, memories, intentions, or other mental states of natural persons on the basis of their biometric or biometrics- based data;
Amendment 295 #
Proposal for a regulation
Article 3 – paragraph 1 – point 34
Article 3 – paragraph 1 – point 34
(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions, thoughts, or intentions of natural persons on the basis of their biometric or biometrics-based data;
Amendment 296 #
Proposal for a regulation
Article 3 – paragraph 1 – point 35
Article 3 – paragraph 1 – point 35
(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, hair colour, eye colour, tattoos, ethnic origin, health, mental ability, behavioural traits, or sexual or political orientation, on the basis of their biometric or biometrics-based data;
Amendment 298 #
Proposal for a regulation
Article 3 – paragraph 1 – point 35 a (new)
Article 3 – paragraph 1 – point 35 a (new)
(35a) ‘biometric inferences’ means conclusions with regards to permanent or long-term physical, physiological, or behavioural characteristics of a natural person, on the basis of biometrics, biometrics-based data, or other personal data;
Amendment 326 #
Proposal for a regulation
Article 5 – paragraph 1 – point c – introductory part
Article 5 – paragraph 1 – point c – introductory part
(c) the placing on the market, putting into service or use of AI systems by public authorities or on their behalf for the evaluation or classification of the trustworthiness of natural persons over a certain period of time based on their social behaviour or known or predicted personal or personality characteristics, with the social score leading to either or both of the following:
Amendment 326 #
(2 a) The deployment of artificial intelligence applications across sectors will only accelerate in the years to come. The European Union should therefore consider, in separate legislation, the creation of an Artificial Intelligence Adjustment Fund, which could be beneficial for Member States to cover the accustoming of their labour markets to the new conditions arising from the rapid mass introduction of artificial intelligence systems that could affect specific job sectors.
Amendment 327 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – introductory part
Article 5 – paragraph 1 – point d – introductory part
(d) the use of ‘real-time’ remoteputting into service or use of biometric or biometrics-based identification systems in publicly accessible spaces for ththat allow the comprehensive or large-scale psurpose of law enforcement, unless and in as far as such use is strictly necessary for one of the following objectives:veillance of natural persons in any context., including surveillance in the workplace.
Amendment 329 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point i
Article 5 – paragraph 1 – point d – point i
Amendment 330 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point ii
Article 5 – paragraph 1 – point d – point ii
Amendment 331 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point iii
Article 5 – paragraph 1 – point d – point iii
Amendment 332 #
Proposal for a regulation
Article 5 – paragraph 1 – point d a (new)
Article 5 – paragraph 1 – point d a (new)
(da) the placing on the market, putting into service, or use of an AI system for the specific technical processing of brain or brain-generated data in order to access, infer, influence, or manipulate a person's thoughts, emotions, memories, intentions, beliefs, or other mental states against that person's will or in a manner that causes or is likely to cause that person or another person physical or psychological harm;
Amendment 334 #
Proposal for a regulation
Article 5 – paragraph 1 – point d b (new)
Article 5 – paragraph 1 – point d b (new)
(db) the putting into service or use of AI systems that allow the categorisation of individuals on the basis of their biometric, biometrics-based data, or biometric inferences into clusters according to ethnicity, gender, political or sexual orientation, or any other grounds that may lead to discrimination prohibited under Article 21 of the Charter of Fundamental Rights of the European Union;
Amendment 336 #
Proposal for a regulation
Article 5 – paragraph 1 – point d c (new)
Article 5 – paragraph 1 – point d c (new)
(dc) The putting into service or use of AI systems for making individual or group assessments of natural persons in order to assess the risk of a natural person or a group of persons for offending or reoffending, or for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of natural persons or on the assessment of personality traits, characteristics, or past criminal behaviour.
Amendment 341 #
Proposal for a regulation
Article 5 – paragraph 2
Article 5 – paragraph 2
Amendment 342 #
Proposal for a regulation
Article 5 – paragraph 2 – point a
Article 5 – paragraph 2 – point a
Amendment 343 #
Proposal for a regulation
Article 5 – paragraph 2 – point b
Article 5 – paragraph 2 – point b
Amendment 345 #
Proposal for a regulation
Article 5 – paragraph 2 – subparagraph 1
Article 5 – paragraph 2 – subparagraph 1
Amendment 346 #
Amendment 348 #
Proposal for a regulation
Article 5 – paragraph 3 – subparagraph 1
Article 5 – paragraph 3 – subparagraph 1
Amendment 349 #
Proposal for a regulation
Article 5 – paragraph 4
Article 5 – paragraph 4
Amendment 350 #
Proposal for a regulation
Article 5 – paragraph 4 a (new)
Article 5 – paragraph 4 a (new)
4a. The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of prohibited practices listed in paragraph 1 of this Article, in order to update that list on the basis of a similar threat to fundamental human rights and values.
Amendment 406 #
Proposal for a regulation
Article 10 – paragraph 2 – point a
Article 10 – paragraph 2 – point a
(a) the relevant design choices; all appliances should be designed with the option to forbid the constantly open microphone/camera of apps and offer consumers a clear option for all recording features to be shut down when the corresponding app is not in use;
Amendment 425 #
Proposal for a regulation
Article 10 – paragraph 3
Article 10 – paragraph 3
3. Training, validation and testing data sets shall be sufficiently relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.
Amendment 429 #
Proposal for a regulation
Article 10 – paragraph 6 a (new)
Article 10 – paragraph 6 a (new)
6a. This Regulation shall guarantee the protection of citizens who choose to lead an "offline life" and ensure that there are always offline options and services available for them, especially when this concerns the provision of essential private and public services.
Amendment 429 #
Proposal for a regulation
Recital 16
Recital 16
(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive, access brain or brain-generated data without consent, or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human- machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research.
Amendment 439 #
Proposal for a regulation
Article 14 – paragraph 1
Article 14 – paragraph 1
1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which the AI system is in usewhole lifecycle of the AI system. AI systems shall not be used to substitute, but rather to complement human decision- making. All AI systems shall be explainable by design.
Amendment 441 #
Proposal for a regulation
Article 14 – paragraph 2
Article 14 – paragraph 2
2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks can affect the wellbeing, health or physical integrity of children and minors, or persist notwithstanding the application of other requirements set out in this Chapter. Special attention shall be paid on AI systems used for the development or as components of children toys.
Amendment 469 #
Proposal for a regulation
Article 15 – paragraph 4 – subparagraph 2
Article 15 – paragraph 4 – subparagraph 2
The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent and control for attacks trying to manipulate the training dataset (‘data poisoning’), inputs designed to cause the model to make a mistake (‘adversarial examples’), or model flaws which could lead to harmful decision-making.
Amendment 595 #
Proposal for a regulation
Article 57 – paragraph 1
Article 57 – paragraph 1
1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authoritiesThe European Union Agency for Fundamental Rights will have the status of observer in the Board. Other national authorities, as well as representatives of small and medium-sized enterprises and startups, may be invited to the meetings, where the issues discussed are of relevance for them.
Amendment 604 #
Proposal for a regulation
Article 58 – paragraph 1 – point c a (new)
Article 58 – paragraph 1 – point c a (new)
Amendment 644 #
Proposal for a regulation
Annex III – paragraph 1 – point 5 – point b
Annex III – paragraph 1 – point 5 – point b
(b) AI systems intended to be used to evaluate the creditworthiness of natural persons or, establish their credit score, or predict medical human conditions and health-related outcomes, with the exception of AI systems put into service by small scale providers for their own use;
Amendment 1284 #
Proposal for a regulation
Article 5 – paragraph 1 – point d a (new)
Article 5 – paragraph 1 – point d a (new)
(d a) the placing on the market, putting into service, or use of an AI system for the specific technical processing of brain or brain-generated data in order to access, infer, influence, or manipulate a person's thoughts, emotions, memories, intentions, beliefs, or other mental states against that person's will or in a manner that causes or is likely to cause that person or another person physical or psychological harm.
Amendment 3135 #
Proposal for a regulation
Annex III – paragraph 1 – point 5 – point b
Annex III – paragraph 1 – point 5 – point b
(b) AI systems intended to be used to evaluate the creditworthiness of natural persons or, establish their credit score, with the exception of AI systems put into service by small scale providers for their own use;or predict human medical conditions and health-related outcomes
Amendment 3241 #
Proposal for a regulation
Annex III – paragraph 1 – point 8 a (new)
Annex III – paragraph 1 – point 8 a (new)
8 a. Others a) AI systems intended to be used for the delivery of online advertising to internet users