59 Amendments of Brando BENIFEI related to 2020/2012(INL)
Amendment 5 #
Draft opinion
Paragraph 1
Paragraph 1
1. Underlines the importance of an EU regulatory framework being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system; furthermore, believes that the rules set out should apply across the value chain, namely development, deployment and use of the relevant technologies and their components and for all developers, and should guarantee the highest level of consumer protection; proposes that these rules take into account the lessons drawn from the adaptation to Regulation (EU) 2016/6791a (GDPR), considered a global benchmark; considers the designation of a liable entity in the Union (such as authorised representative) important for its enforcement; __________________ 1aRegulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1).
Amendment 9 #
Draft opinion
Paragraph 2
Paragraph 2
2. Notes that the framework should apply to algorithmic systems, including the fields of artificial intelligence (AI), internet of things (IoT), machine learning, deep learning, automated decision making processes and robotics; further notes that referral systems should be developed to help explain those systems to consumers whenever they present complexity or constitute decisions that impact their lives significantly;
Amendment 14 #
Draft opinion
Recital B
Recital B
B. Wwhereas AI potentially offers economic and societal benefits, while at the same time raising a number of challengesfor example with regard to fighting inequalities and improving the quality of life, while at the same time raising a number of challenges; whereas the application of AI at the workplace can help to improve occupational health and safety, while it can also be used to monitor, evaluate, predict and steer the performance of workers with direct and implicit consequences for their careers and on their mental well-being;
Amendment 21 #
Draft opinion
Paragraph 3
Paragraph 3
3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm or breaches of rights for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations and certification requirements should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual and certification or labelling systems should be voluntary; algorithmic systems that may harm an individual or cause potential breaches of an individual’s rights, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules, providing enough legal certainty whilst being future-proof;
Amendment 28 #
Draft opinion
Recital C a (new)
Recital C a (new)
Ca. whereas 42% of workers in companies that apply AI in their business processes believe that such activities lead to ethical issues, which must be addressed; whereas 28% of the employers believe that the application of AI has not developed at full scale because of a lack of ethical rules on this issue1a; _________________________ 1aCapgeminiResearch Institute, “Why addressing ethical questions in AI will benefit organizations”, July 2019
Amendment 31 #
Draft opinion
Paragraph 4
Paragraph 4
4. Underlines the importance of an ethical and regulatory framework, including in particular provisions on the quality of data sets used in algorithmic systems in relation to the purpose of its use, especially regarding the representativeness of training data used, on the de-biasing of data sets, as well as on the algorithms themselves, and on data and aggregation standards; stresses that those data sets should be auditable and made available to the competent authorities whenever called upon to ensure their fitness with the previously exposed principles;
Amendment 32 #
Draft opinion
Recital C b (new)
Recital C b (new)
Cb. whereas the OECD has drawn up recommendations on artificial intelligence1a; __________________ 1aOECD, “Recommendation of the Council on Artificial Intelligence”, 2019, https://legalinstruments.oecd.org/en/instr uments/OECD-LEGAL-0449
Amendment 34 #
Draft opinion
Recital C c (new)
Recital C c (new)
Cc. whereas the Council of the European Union encourages the promotion of an ethical and human- centred approach with regard to AI1a; __________________ 1aCouncil of the European Union “Council Conclusions on Shaping Europe’s Digital future”, June 2020
Amendment 36 #
Draft opinion
Recital C d (new)
Recital C d (new)
Cd. whereas there are no common provisions at Union level, as regards the application of AI at the workplace, which could lead to market distortions and competition disadvantages; whereas unified rules are necessary to provide workers and companies with a reliable regulatory framework;
Amendment 37 #
Draft opinion
Recital C e (new)
Recital C e (new)
Ce. whereas social partners at Union level concluded a framework agreement on digitalisation, which amongst others includes a chapter on “Artificial intelligence and guaranteeing the human in control principle” 1a; __________________ 1a European Social Partners Framework Agreement on Digitalisation, June 2020
Amendment 38 #
Draft opinion
Recital C f (new)
Recital C f (new)
Cf. whereas some Member States have already established special bodies to monitor and assess the influence of AI at the workplace;
Amendment 39 #
Draft opinion
Recital C g (new)
Recital C g (new)
Cg. whereas workers and their representatives are often neither aware of AI applications nor the underlying functions and data; whereas lacking awareness and competitive pressure led to some companies applying AI in breach of existing regulations, such as data protection;
Amendment 40 #
Draft opinion
Recital C h (new)
Recital C h (new)
Ch. whereas efforts to tackle gender bias and inequality in the digital sector are insufficient; whereas the gender gap persists across all digital technology domains and especially with regard to AI, thereby solidifying a male-biased trajectory for the digital sector in the foreseeable future;
Amendment 45 #
Draft opinion
Paragraph 5
Paragraph 5
5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possibleintended and reasonably foreseeable unintended outcome of algorithmic systems, about how to reach a human with decision- making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; this information should not, however, endanger trade secrets and other competitive advantages;
Amendment 46 #
Draft opinion
Paragraph 5
Paragraph 5
5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision- making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; recalls that humans must always be able to overrule automated decisions that are final and permanent;
Amendment 50 #
Draft opinion
Paragraph 1
Paragraph 1
1. Highlights the need to thoroughly assess the effects or implications of AI applications in companies and in public administration in relation to workers, jobs and workflows; considers it indispensable as part of this assessment that workers and their representatives are consulted and receive sufficient information before AI applications are put to use; underlines that the deployment of AI needs to be transparent and that AI systems at the workplace must not undermine the privacy and dignity of workers;
Amendment 61 #
Draft opinion
Paragraph 2
Paragraph 2
2. Points out that AI solutions have the potential to improve working conditions and the quality of life, yet they can also lead to disproportionate and illegal surveillance of workers, discriminatory treatment due to biased algorithms, including gender or ethnically biased algorithms3 - and they can undermine the dignity and autonomy of peopland algorithms that are to the detriment of vulnerable groups and that undermine the privacy and dignity of workers; is concerned, furthermore, that AI can undermine the dignity and autonomy of workers and contribute to mental health problems, such as burnout, occupational stress, psychological overload and fatigue; __________________ 3 European Parliament “Education and employment of women in science, technology and the digital economy, including AI and its influence on gender equality”, April 2020
Amendment 63 #
Draft opinion
Paragraph 7
Paragraph 7
7. Stresses that where public money contributesmoney originating from public sources contributes significantly to the development or implementation of an algorithmic system, the code, the generated data - as far as it is non-personal - and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;
Amendment 67 #
Draft opinion
Paragraph 2 – subparagraph 1 (new)
Paragraph 2 – subparagraph 1 (new)
Underlines that competent authorities should have access to all information concerning the data used for training, statistical models and theoretical principles related to AI solutions as well as the empirical validity of their outcomes;
Amendment 69 #
Draft opinion
Paragraph 7 a (new)
Paragraph 7 a (new)
7a. Underlines that the increased use of artificial intelligence requires a strong focus on digital security, as the large amount of data creates new risks of cyberattacks; calls on the Commission to develop clear guidelines for businesses and public agencies to take the necessary precautions when using artificial intelligence;
Amendment 75 #
Draft opinion
Paragraph 7 b (new)
Paragraph 7 b (new)
7b. Considers that AI, IoT, and other emerging technologies have enormous potential to deliver opportunities for consumers to have access to several amenities in many aspects of their lives alongside with better products and services, as well as to benefit from better market surveillance, as long as all applicable principles, conditions (including transparency and auditability), and regulations continue to apply;
Amendment 80 #
Draft opinion
Paragraph 3
Paragraph 3
3. Considers that it should be mandatory for users and consumers to receive all relevant information on the ethical aspects of AI applications so that they may mto take informed decisions;
Amendment 81 #
Draft opinion
Paragraph 8
Paragraph 8
8. Underlines the importance of ensuring that the interests of consumers and groups which are marginalised andor in vulnerable consumers and groupsituations are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;
Amendment 81 #
Draft opinion
Paragraph 3 – subparagraph 1 a (new)
Paragraph 3 – subparagraph 1 a (new)
Underlines that trustworthy AI has to be fair, transparent, safe and secure and to comply with all applicable laws and regulations including the General Data Protection Regulation (GDPR) throughout the system’s entire life cycle, especially when it is deployed at the workplace;
Amendment 86 #
Motion for a resolution
Paragraph 2 a (new)
Paragraph 2 a (new)
2a. Underlines that the risk assessment of these technologies in light of ethical principles shall be complementary but not replace the risk assessment on the basis of objective criteria and in line with relevant sector- specific legislation applicable in different fields such, inter alia, as those of health, transport, employment, justice and home affairs, media, education and culture;
Amendment 88 #
Motion for a resolution
Paragraph 2 b (new)
Paragraph 2 b (new)
2b. Considers that any artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies considered as high-risk from a compliance with ethical principles perspective shall not be classified as low- risk from a sector-specific legislation perspective;
Amendment 93 #
Draft opinion
Paragraph 4
Paragraph 4
4. Stresses the need to ensure that productivity gains due to the development and use of AI and robotics do not only benefit company owners and shareholders, but also society at large, especially where such gains come at the expense of jobs; underlines that companies deploying AI have the responsibility to provide the necessary re- and upskilling for the employees concerned;
Amendment 99 #
Draft opinion
Paragraph 10
Paragraph 10
10. Calls for the Union to establishment of a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities;n the framework of a European Agency for Artificial Intelligence, which the Parliament proposed in its resolution of 16 February 20171a, issuing guidance, opinions and expertise to Member States’ authorities, monitoring the implementation of relevant EU legislation, addressing potential consumer protection issues, identifying standards for best practice, and, where appropriate, making recommendations for regulatory measures; __________________ 1aEuropean Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics (2015/2103(INL)) (OJ C 252, 18.7.2018, p. 239).
Amendment 100 #
Draft opinion
Paragraph 5
Paragraph 5
5. Calls for the application of the precautionary principle with regard to new technologies based on AI; underlines that humans must always be in control of machines and AI and that AI decisions must be reversiblecontestable and reversible; stresses that safety and security standards for AI must be respected and the importance of regular checks and controls in this regard to prevent erroneous AI output;
Amendment 102 #
Draft opinion
Paragraph 10
Paragraph 10
10. Calls for the Union to establish a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities; emphasizes that Member States must develop risk-management strategies for AI in the context of their national market surveillance strategies.
Amendment 106 #
Draft opinion
Paragraph 5 a (new)
Paragraph 5 a (new)
5a. Stresses that applicants must be duly informed in writing in case AI is used in the course of recruitment procedures and how in this case a human review can be requested in view of reversing an automated decision;
Amendment 108 #
Draft opinion
Paragraph 11
Paragraph 11
11. Notes that it is essential for the software documentation, the algorithms and data sets used or produced by artificial intelligence, robotics, and related technologies to be fully accessible to market surveillance authorities, while respecting Union law; further notes that such elements should be preserved by those who are involved in the different stages of the development of algorithmic systems and in proportion of their liability, namely through distributed ledger technologies, such as block-chain; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect;
Amendment 116 #
Draft opinion
Paragraph 12
Paragraph 12
12. Calls for the designation by each Member State of a competent national authorityenforcement body(NEB) for monitoring the application of the provisions;
Amendment 117 #
Draft opinion
Paragraph 6 – subparagraph 1a (new)
Paragraph 6 – subparagraph 1a (new)
Stresses that the Skills Agenda for Europe must address the challenges of adapting and acquiring skills and knowledge, in view of the ecological and digital transition, including ethical aspects of AI, robotics and related technologies; underlines the need to make ethical aspects of AI an integral part of any education and training curricular for developers and people working with AI; considers it likewise important to ensure the comprehensive information of end users and consumers in this regard;
Amendment 120 #
Draft opinion
Paragraph 6 a (new)
Paragraph 6 a (new)
6a. Stresses that developers, programmers, decision-makers and companies providing AI and robotics solutions must be aware of their ethical responsibility when it comes to providing AI and robotics solutions;
Amendment 123 #
Draft opinion
Paragraph 6 b (new)
Paragraph 6 b (new)
6b. Underlines that AI must not in any way affect the exercise of fundamental rights as recognised in the Member States and at Union level, including the right or freedom to strike or to take other action covered by the specific industrial relations systems in Member States, in accordance with national law and/or practice, nor affect the right to negotiate, to conclude and enforce collective agreements, or to take collective action in accordance with national law and/or practice;
Amendment 125 #
Draft opinion
Paragraph 13
Paragraph 13
13. Calls for the establishment of a European market surveillance board for algorithmic systems in the framework of a European Agency for Artificial Intelligence, to ensure a level playing field and to avoid fragmentation of the internal market, to decide with a qualified majority and by secret vote in case of different decisions on algorithmic systems used in more than one Member State, as well as at the request of the majority of the national authorities;
Amendment 125 #
Draft opinion
Paragraph 6 c (new)
Paragraph 6 c (new)
6c. Calls for a legislative framework on the ethical aspects of AI applications at the workplace, in order to ensure a level- playing field and fair competition in the Single Market, especially with regard to workers’ rights as well as working conditions and the protection from disproportionate and illegal surveillance; calls on the Commission to consult with social partners and other stakeholders such as researchers and developers in this regard as well to explore the potential of digital technology and AI to increase the well-being of the workforce, including a better allocation of tasks, augmented competence development and work capacities as well as the reduction of exposure to harmful working conditions;
Amendment 126 #
Draft opinion
Paragraph 6 d (new)
Paragraph 6 d (new)
6d. Underlines that special attention must be payed to data collected at the workplace with the help of AI in so far as it is used for human resources decisions; calls on the Commission to analyse the need for special provisions on data protection at the workplace in the context of AI; stresses that workers must be the owners of their data;
Amendment 127 #
Draft opinion
Paragraph 6 e (new)
Paragraph 6 e (new)
6e. Underlines that AI must not reinforce gender inequalities and stereotypes by transforming analogue biases and prejudices into digital ones through algorithms;
Amendment 129 #
Draft opinion
Paragraph 13 a (new)
Paragraph 13 a (new)
13a. Stresses the need for national market surveillance authorities to be reinforced in terms of capacity, skills, and competences in AI, IoT and other related technologies, as well as knowledge about its specific risks;
Amendment 170 #
Motion for a resolution
Paragraph 16 a (new)
Paragraph 16 a (new)
16a. Considers that a common European framework for the governance of these technologies prepared by the European Commission or another relevant institution, body, office or agency of the Union (such as a European Agency for Artificial Intelligence, as called for below) designated for this task should be the basis for this governance in order to ensure a coherent European approach and prevent a fragmentation of the single market;
Amendment 175 #
Motion for a resolution
Paragraph 17 a (new)
Paragraph 17 a (new)
17a. Recalls that access to data is an essential component in the growth of the digital economy; points out in this regard that interoperability of data, by limiting lock-in effects, plays a key role in ensuring fair market conditions and promoting a level playing field in the Digital Single Market;
Amendment 189 #
Motion for a resolution
Paragraph 19 a (new)
Paragraph 19 a (new)
19a. Notes that these authorities shall cooperate with the authorities responsible for implementing sectorial legislation in order to identify technologies which are high-risk from an ethical perspective and in order to supervise the implementation of required and appropriate measures where such technologies are identified;
Amendment 216 #
Motion for a resolution
Paragraph 23
Paragraph 23
23. Calls on the Commission to follow- up on that request, especially in view of the added-value of having a body at Union level coordinating the mandates and actions of each national supervisory authority as referred to in the previous sub- section and drafting a common framework for the governance of the development, deployment and use of artificial intelligence, robotics and related technologies;
Amendment 334 #
Motion for a resolution
Annex I – part A – point V – indent 3 a (new)
Annex I – part A – point V – indent 3 a (new)
- to draft a common framework for the governance of the development, deployment and use of artificial intelligence, robotics and related technologies to be implemented by the national supervisory authorities;
Amendment 352 #
Motion for a resolution
Annex I – part A – point VI – indent 4
Annex I – part A – point VI – indent 4
- to be responsible for establishimplementing standards for the governance of artificial intelligence, robotics and related technologies, including by liaising with the maximum possible number of stakeholders and civil society representatives.
Amendment 353 #
Motion for a resolution
Annex I – part A – point VI – indent 4 a (new)
Annex I – part A – point VI – indent 4 a (new)
- to cooperate with the authorities responsible for implementing sectorial legislation in order to identify technologies which are high-risk from an ethical perspective and in order to supervise the implementation of required and appropriate measures where such technologies are identified;
Amendment 383 #
Motion for a resolution
Annex I – part B – recital 7
Annex I – part B – recital 7
(7) The development, deployment and use of artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies, should be such as tomeant to complement human capabilities, not to substitute them, it should be aimed at ensureing that the best interests of citizens are considered, and it should respect fundamental rights as set out in the Charter of Fundamental Rights of the European Union (‘the Charter’), settled case-law of the Court of Justice of the European Union, and other European and international instruments which apply in the Union.
Amendment 400 #
Motion for a resolution
Annex I – part B – recital 10
Annex I – part B – recital 10
(10) Notwithstanding the risk assessment carried out in relation to compliance with ethical principles, artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies, should always be assessed as to their risk and subject to corresponding legal obligations on the basis of objective criteria and in line with relevant sector-specific legislation applicable in different fields such as those of health, transport, employment, justice and home affairs, media, education and culture.
Amendment 402 #
Motion for a resolution
Annex I – part B – recital 10 a (new)
Annex I – part B – recital 10 a (new)
(10a) Any artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies considered as high-risk from a compliance with ethical principles perspective shall not be classified as low risk from a sector- specific legislation perspective.
Amendment 423 #
Motion for a resolution
Annex I – part B – recital 20
Annex I – part B – recital 20
(20) In line with Union law, legitimate aims that might objectively justify any differential treatment between persons or group of persons are the protection of public safety, security and health, the prevenotection of criminal offences, the protection of individufundamental rights and freedoms, fair representation and objective professional requirements.
Amendment 462 #
Motion for a resolution
Annex I – part B – recital 38
Annex I – part B – recital 38
(38) The effective application of the ethical principles laid down in this Regulation will largely depend on Member States’ appointment of an independent public authority to act as a supervisory authority. In particular, each national supervisory authority should be responsible for assessing and monitoring the compliance ofidentifying artificial intelligence, robotics and related technologies considered a high-risk and for assessing and monitoring the compliance of these technologies in light of the obligations set out in this Regulation.
Amendment 464 #
Motion for a resolution
Annex I – part B – recital 39
Annex I – part B – recital 39
(39) Each national supervisory authority shall also carry the responsibility of regulating the governance of these technologies in accordance with a common framework prepared by the European Commission or another relevant institution, body, office or agency of the Union designated for this task. They therefore have an important role to play in promoting the trust and safety of Union citizens, as well as in enabling a democratic, pluralistic and equitable society.
Amendment 467 #
Motion for a resolution
Annex I – part B – recital 39 a (new)
Annex I – part B – recital 39 a (new)
(39a) National supervisory authorities shall cooperate with the authorities responsible for assessing and monitoring artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies and enforcing their compliance in the light of sectorial legislation in order to identify technologies which are high-risk from an ethical perspective and in order to supervise the implementation of required and appropriate measures where such technologies are identified;
Amendment 483 #
Motion for a resolution
Annex I – part B – recital 46
Annex I – part B – recital 46
(46) Action at Union level as set out in this Regulation would be best achieved through the establishment of a European Agency for Artificial Intelligence. Such a body would be essential in coordinating the mandates and actions of the national supervisory authorities in each Member State, outlining objective criteria for the risk assessment of artificial intelligence, robotics and related technologies, drafting a common framework for the governance of the development, deployment and use of these technologies, developing and issuing a certification of compliance with the ethical principles laid down in this Regulation, supporting regular exchanges with concerned stakeholders and civil society, promoting the Union’s approach through international cooperation and ensuring a consistent reply worldwide to the opportunities and risks inherent in these technologies.
Amendment 625 #
Motion for a resolution
Annex I – part B – Article 13 – paragraph 1
Annex I – part B – Article 13 – paragraph 1
1. Artificial intelligence, robotics and related technologies developed, deployed or used in the Union shall comply with relevant governance standards establishdrafted by the European Commission or another relevant institution, body, office or agency of the Union designated for this task and implemented by the national supervisory authorities referred to in Article 14 in accordance with Union law, principles and values.
Amendment 634 #
Motion for a resolution
Annex I – part B – Article 14 – paragraph 1
Annex I – part B – Article 14 – paragraph 1
1. Each Member State shall designate an independent public authority to be responsible for monitoring the application of this Regulation (‘supervisory authority’). In accordance with Article 7(1) and (2), each national supervisory authority shall be responsible for assessing whether artificial intelligence, robotics and related technologies, including software, algorithms and data used or produced by such technologies, developed, deployed and used in the Union are high-risk technologies in view of the ethical principles set out in this Regulation and, if so, for assessing and monitoring their compliance with the ethicalse principles set out in this Regulation.
Amendment 635 #
Motion for a resolution
Annex I – part B – Article 14 – paragraph 1 a (new)
Annex I – part B – Article 14 – paragraph 1 a (new)
1a. Each national supervisory authority shall supervise the implementation of the legislation in force and, where applicable, the enforceable sectorial regulations arising when a technology is identified as a high-risk technology in view of the ethical principles set out in this Regulation. For this purpose, the supervisory authorities shall cooperate with the sectorial authorities in charge of assessing and monitoring artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies and enforcing their compliance in the light of sectorial legislation.