人工智能特朗普回应美国确诊超百万

国家人工智能研发战略计划:2019年更新一份报告选择人工智能委员会的国家科学技术委员会2019年6月亲爱的同事们,特朗普总统在2019年2月5日的国情咨文演讲中强调了确保美国在开发构成未来产业的新兴技术(包括人工智能)方面发挥领导作用的重要性.
为了反映这一重要性,特朗普总统于2019年2月11日签署了行政命令13859,该命令建立了美国人工智能倡议.
该倡议是一种整体政府方法,用于维持美国在人工智能方面的领导地位,确保人工智能使美国人民受益,并反映我们国家的价值观.
本行政命令中的第一条指令是联邦机构在其年度预算和规划过程中优先考虑人工智能研究与开发(R&D).
随附的国家AI研发战略计划:2019年更新突出了联邦投资人工智能研发的主要优先事项.
人工智能提供了巨大的机会,导致改善医疗保健,更安全和更有效的运输,个性化教育,重大科学发现,改进制造,提高农业作物产量,更好的天气预报等方面的突破.
这些好处主要归功于几十年来联邦对基础人工智能研发的长期投资,这些投资为人工智能系统带来了新的理论和方法,以及允许将人工智能转化为实际应用的应用研究.
由于行业,学术界和非营利组织正在进行大量投资,人工智能研发的前景正变得越来越复杂.
此外,人工智能的进步正在迅速发展.
因此,联邦政府必须不断重新评估其人工智能研发投资的优先级,以确保投资继续推进该领域的前沿,并且不会不必要地重复行业投资.
在2018年8月,政府指示人工智能专责委员会更新2016年国家人工智能研发战略计划.
这一过程始于发布信息请求,以征求公众对如何修订或改进战略的意见.
对此RFI的回复以及独立的机构审查将此更新告知了战略计划.
在本战略计划中,确定了八个战略优先事项.
前七个策略继续从2016年计划开始,反映了公众和政府的多个受访者重申这些策略的重要性,没有要求删除任何策略.
第八项战略是新的,重点关注联邦政府与学术界,工业界,其他非联邦实体和国际盟友之间有效伙伴关系日益增加的重要性,以便在人工智能方面取得技术突破并迅速将这些突破转化为能力.
虽然该计划没有为联邦机构投资确定具体的研究议程,但它确实为联邦人工智能研发投资的整体投资组合提供了期望.
这项协调的联邦人工智能研发战略将帮助美国继续在人工智能的前沿领先世界,引领我们的经济,增强国家安全,提高生活质量.
此致米迦勒·克瑞西俄斯技术政策总裁助理助理2019年6月21日国家人工智能研发战略计划:2019–ii–目录执行摘要iii2019年国家AI研发战略计划简介1人工智能研发战略.
5战略1:对人工智能研究进行长期投资.
72019年更新:持续对基础AI研究进行长期投资7推进以数据为中心的知识发现方法.
9增强AI系统的感知能力.
9了解AI的理论能力和局限性.
10开展通用人工智能研究10开发可扩展的AI系统.
11促进人类AI的研究.
11开发更强大,更可靠的机器人.
11推进硬件以改进AI.
12创建AI以改进硬件.
12战略2:开发有效的人工智能协作方法.
142019年更新:开发人工智能系统,补充和增强人类能力,并不断增加专注于工作的未来.
14寻求人类感知AI的新算法.
17开发用于人体增强的AI技术.
17开发可视化和人工智能接口技术18开发更有效的语言处理系统.
18策略3:理解并解决人工智能的道德,法律和社会影响192019年更新:解决人工智能中的道德,法律和社会问题.
19通过设计提高公平性,透明度和问责制21建立道德AI.
21为道德AI设计架构.
21战略4:确保AI系统的安全性232019年更新:创建健壮且值得信赖的AI系统.
23提高可解释性和透明度25建立信任.
25加强验证和验证25防范攻击.
26实现长期人工智能安全和价值调整.
26策略5:为人工智能培训和测试开发共享的公共数据集和环境272019年更新:增加对数据集和相关挑战的访问.
27开发和提供各种数据集,以满足各种需求人工智能的兴趣和应用.
29根据商业和公共利益制定培训和测试资源.
30开发开源软件库和工具包.
30策略6:通过标准和基准测量和评估AI技术322019年更新:支持开发人工智能技术标准和相关工具32制定广泛的AI标准.
33建立AI技术基准34增加AI测试平台的可用性.
34让AI社区参与标准和基准测试.
35战略7:更好地了解国家AI研发人员的需求.
372019年更新:推动AI研发人员,包括那些从事AI系统和工作的人员与他们一起,维持美国的领导地位37战略8:扩大公私合作伙伴关系,加速人工智能的发展40缩略语43国家人工智能研发战略计划:2019–iii–执行摘要人工智能(AI)拥有巨大的希望,几乎可以使社会的各个方面受益,包括经济,医疗保健,安全,法律,运输,甚至技术本身.
2019年2月11日,总统签署了行政命令13859,维持美国在人工智能方面的领导地位.
1该订单启动了美国人工智能倡议,这是一项促进和保护美国人工智能技术和创新的共同努力.
该倡议与私营部门,学术界,公众和志同道合的国际合作伙伴合作实施全政府战略.
在其他行动中,该倡议的关键指令要求联邦机构优先考虑人工智能研究与开发(R&D)投资,增强对高质量网络基础设施和数据的访问,确保国家在制定人工智能技术标准方面处于领先地位,并提供为新的人工智能时代准备美国劳动力的教育和培训机会.
为支持美国人工智能倡议,该国家人工智能研发战略计划:2019年更新确定了联邦投资于人工智能研发的优先领域.
2019年的更新建立在2016年发布的第一个国家AI研发战略计划的基础上,考虑了过去三年中出现的新研究,技术创新和其他考虑因素.
此更新由来自联邦政府的领先AI研究人员和研究管理人员开发,得到了更广泛的民间社会的投入,包括来自美国许多领先的学术研究机构,非营利组织和私营部门技术公司.
这些主要利益攸关方的反馈肯定了2016年战略计划各部分的持续相关性,同时也呼吁更加重视人工智能值得信赖,与私营部门合作以及其他必要措施.
国家人工智能研发战略计划:2019年更新为联邦政府资助的人工智能研究制定了一系列目标,确定了以下八个战略重点:策略1:对人工智能研究进行长期投资.
优先考虑对下一代人工智能的投资,这将推动发现和洞察,并使美国成为人工智能的世界领导者.
战略2:为人工智能协作开发有效的方法.
增加对如何理解创建有效补充和增强人类能力的AI系统.
策略3:理解并解决人工智能的道德,法律和社会影响.
通过技术机制研究人工智能系统,包括道德,法律和社会问题.
策略4:确保AI系统的安全性.
了解如何设计可靠,可靠,安全和值得信赖的AI系统.
策略5:为AI培训和测试开发共享的公共数据集和环境.
开发并实现对高质量数据集和环境的访问,以及测试和培训资源.
战略6:通过标准和基准测量和评估AI技术.
为人工智能开发广泛的评估技术,包括技术标准和基准.
策略7:更好地了解国家AI研发人员的需求.
改善研发劳动力发展的机会,从战略上培养一支适合AI的劳动力队伍.
战略8:扩大公私伙伴关系,加速人工智能的发展.
与学术界,行业,国际合作伙伴和其他非联邦实体合作,促进人工智能研发持续投资和将进步转化为实际能力的机会.
1https://www.
whitehouse.
gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/国家人工智能研发战略计划:2019–5–2019年国家AI研发战略计划简介人工智能使计算机和其他自动化系统能够执行历史上需要人类认知的任务以及我们通常认为的人类决策能力.
在过去的几十年里,人工智能已经取得了巨大的进步,今天承诺提供更好,更准确的医疗服务;加强国家安全;改善运输;更有效的教育,仅举几例.
增强的计算能力,大型数据集和流数据的可用性以及机器学习(ML)的算法进步使人工智能开发成为可能,创造了经济的新部门并振兴了行业.
随着越来越多的行业采用人工智能的基础技术,该领域将继续在全球范围内推动深刻的经济影响和生活质量改善.
这些进步主要得益于联邦对人工智能研发的投资,美国无与伦比的研发机构的专业知识,以及许多美国最具远见的科技公司和企业家的集体创造力.
2016年,联邦政府发布了第一份国家人工智能研发战略计划,认识到人工智能的巨大希望和持续发展的需要.
它的开发是为了指导国家的人工智能研发投资,为改善和利用美国的人工智能能力提供战略框架,并确保这些能力在未来几年为美国人民带来繁荣,安全和提高生活质量.
该计划为投资人工智能的联邦机构确定了几个重点关注的重点领域.
这些重点领域或战略包括:对人工智能的持续长期投资;有效的人工智能协作方法;理解和解决人工智能的道德,法律和社会影响;确保AI的安全性;开发用于AI培训和测试的共享公共数据集和环境;通过标准和基准测量和评估AI技术;并更好地了解国家的人工智能研发2https://www.
nitrd.
gov/news/RFI-National-AI-Strategic-Plan.
aspx3https://www.
nitrd.
gov/nitrdgroups/index.
phptitle=AI-RFI-Responses-20182019更新RFI响应通知2019年国家人工智能研发战略计划2018年9月,国家网络和信息技术研究与发展协调办公室发布了信息请求(RFI)2代表人工智能特别委员会,要求所有相关方就2016年国家人工智能研究与发展战略计划提出意见.
研究人员,研究组织,专业协会,民间社会组织和个人提交了近50份答复;这些回复可在线获取.
3许多回复重申了2016年国家AI研发战略计划中概述的分析,组织和方法.
大量答复指出了在制造业和供应链等领域投资人工智能的重要性;卫生保健;医学影像;气象学,水文学,气候学和相关领域;网络安全;教育;数据密集型物理科学,如高能物理学;和运输.
自2016年国家AI研发战略计划发布以来,人们对AI技术的转化应用的兴趣肯定有所增加.
RFI响应中回应的其他共同主题是开发值得信赖的AI系统的重要性,包括公平,道德,问责制和AI系统的透明度;策划和可访问的数据集;劳动力考虑;以及促进人工智能研发的公私合作伙伴关系.
国家人工智能研发战略计划:2019–6–劳动力需求.
这项工作具有先见之明:今天,世界各国都纷纷效仿并发布了自己的计划版本.
自国家人工智能研发战略计划制定以来的三年中,新研究,技术创新和现实部署迅速发展.
行政当局启动了2019年国家人工智能研发战略计划的更新,以解决这些进步,包括快速发展的国际人工智能环境.
值得注意的是,2019年国家人工智能研发战略计划更新的设计仅涉及解决与推进人工智能技术相关的研究和开发优先事项.
它没有描述或推荐与人工智能治理或部署相关的政策或监管行动,尽管人工智能研发肯定会为制定合理的政策和监管框架提供信息.
AI作为管理优先级自2017年以来,行政当局通过强调其在多个主要政策文件(包括国家安全战略)中对美国未来的作用,阐述了人工智能研发的重要性.
4国防战略,5和2020财年研发预算优先事项备忘录.
62018年5月,科技政策办公室(OSTP)主办了白宫人工智能美国产业峰会,开始讨论人工智能的承诺以及实现美国人民承诺并保持美国领导地位所需的政策.
人工智能的年龄.
峰会召集了100多位政府高级官员,来自顶级学术机构的技术专家,工业研究实验室负责人和美国商界领袖.
特朗普总统在2019年2月5日的国情咨文演讲中强调了确保美国在发展构成未来产业的新兴技术(包括人工智能)方面发挥领导作用的重要性.
2019年2月11日,总统签署了行政命令13859,维持美国在人工智能方面的领导地位.
7该订单启动了美国人工智能倡议,这是一项促进和保护美国人工智能技术和创新的共同努力.
该倡议通过与私营部门,学术界,公众和志同道合的国际合作伙伴的合作和参与实施整体政府战略.
在其他行动中,该倡议的关键指令要求联邦机构优先考虑人工智能研发投资,增加对高质量网络基础设施和数据的访问,确保国家在制定人工智能技术标准方面处于领先地位,并提供教育和培训机会.
为AI新时代的美国劳动力做好准备.
制定2019年国家人工智能研发战略计划更新2016年国家人工智能研发战略计划建议,负责推进或采用人工智能的许多联邦机构合作确定关键的研发机会,并支持联邦人工智能研发活动的有效协调,包括校内和校外研究.
国家科学技术委员会(NSTC)反映了政府对人工智能的优先顺序,建立了一个新的框架来实施这一建议,由三个独特的NSTC小组组成,这些小组由来自联邦研发机构的成员组成,涵盖(1)高级领导层4https://www.
whitehouse.
gov/wp-content/uploads/2017/12/NSS-Final-12-18-2017-0905.
pdf5https://dod.
defense.
gov/Portals/1/Documents/pubs/2018-National-Defense-Strategy-Summary.
pdf6https://www.
whitehouse.
gov/wp-content/uploads/2018/07/M-18-22.
pdf7https://www.
whitehouse.
gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/国家人工智能研发战略计划:2019–7–和战略愿景,(2)运营规划和战术实施,(3)研究和技术专长.
这些小组是:人工智能专题委员会,8由主要负责政府人工智能研发的部门和机构负责人组成,就机构间人工智能研发优先事项向政府提出建议;考虑与工业界和学术界建立联邦伙伴关系;建立结构,以改善政府规划和协调人工智能研发;确定利用联邦数据和计算资源来支持我们的国家AI研发生态系统的机会;并支持技术,国家人工智能劳动力的增长.
NSTC机器学习和人工智能小组委员会(MLAI)由机构人工智能领导和管理人员组成,是专责委员会的运作和实施部门,负责完成特别委员会的任务;创建和维护国家AI研发战略计划;确定和解决与人工智能研究,测试,标准,教育,实施,外联和相关领域相关的重要政策问题;和相关活动.
人工智能研发机构间工作组在NSTC的网络和信息技术研发(NITRD)小组委员会下运作,由来自联邦政府的研究项目经理和技术专家组成,向MLAI小组委员会报告;帮助协调跨机构AI研发计划的工作;作为跨机构AI研发实践社区;并通过NITRD小组委员会年度总统预算补编报告政府范围内的人工智能研发支出.
在2018年9月,专责委员会启动了对2016年战略计划的更新,首先是RFI寻求广泛的社群意见,了解2016年国家AI研发战略计划的七项战略是否以及如何值得修改或更换(见附文).
独立地,执行或资助AI研发的联邦部门和机构进行了自己的评估.
2019年国家人工智能研发战略计划更新概述人工智能专题委员会,NSTC机器学习和人工智能小组委员会以及NITRD人工智能研发机构间工作组共同审查了有关国家人工智能研发战略计划的意见.
2016年计划的最初七个重点领域或战略中的每一个都得到了公众和政府的多个受访者的重申,并没有要求删除任何一项战略.
这些战略在2019年战略计划更新中更新,以反映当前的最新技术水平,包括:战略1:对人工智能研究进行长期投资;战略2:制定有效的人工智能合作方法;策略3:理解并解决AI的道德,法律和社会影响;战略4:确保AI系统的安全性;战略5:为人工智能培训和测试开发共享的公共数据集和环境;战略6:通过标准和基准测量和评估AI技术;和战略7:更好地了解国家AI研发人员的需求.
8https://www.
whitehouse.
gov/wp-content/uploads/2018/05/Summary-Report-of-White-House-AI-Summit.
pdf国家人工智能研发战略计划:2019–8–鉴于私人资助的人工智能研发的快速增长以及工业界对人工智能的快速采用,许多对RFI的回应呼吁加强联邦政府对私营部门的研发投入.
因此,2019年更新采用了新的第八个战略:战略8:扩大公私伙伴关系,加速人工智能的发展.
公众和联邦机构的反馈意见确定了进一步促进人工智能开发和采用的若干具体挑战.
这些挑战(其中许多涉及多个机构)提供了更深入的洞察,这些国家AI研发战略计划可以指导美国人工智能研发的过程,并且许多与2019年维护美国领导力执行令的主题密切相关.
在人工智能.
示例包括以下内容:边疆研究.
尽管机器学习在过去几年中带来了惊人的新功能,但仍需要继续研究以进一步推动ML的前沿,并开发其他方法来应对AI的严峻技术挑战(战略1).
积极影响.
随着人工智能能力的增长,美国必须更加重视开发新的方法,以确保人工智能的影响对未来具有强烈的积极作用(战略1,3和4).
信任和可解释性.
真正值得信赖的人工智能需要可解释的人工智能,特别是随着人工智能系统规模和复杂性的增长这需要人类用户和人类设计者全面了解AI系统(策略1,2,3,4和6).
安全保障.
研究人员必须设计出保持AI系统及其使用的数据安全的方法,以便国家能够利用这项技术提供的机会,同时保持机密性和安全性(战略4,5和6).
技术标准.
随着国家开发技术扩展AI能力和保证,它必须测试和基准测试;当技术准备就绪时,它们应该转变为世界的技术标准(战略6).
劳动力能力.
实现这些目标需要培养目前有限且需求量大的熟练人工智能研发人员队伍;美国必须具有创造力和勇气,在培训和获得领导全球人工智能研究和应用所需的熟练劳动力方面(战略7).
合作伙伴关系.
人工智能研发的进步越来越需要联邦政府与学术界,工业界和其他非联邦实体之间建立有效的伙伴关系,以便在人工智能中产生技术突破,并迅速将这些突破转化为能力(战略8).
与盟国合作.
此外,该计划认识到国际合作对于成功实现这些目标的重要性,同时保护美国AI研发企业免受战略竞争对手和敌对国家的影响.
2019年国家AI研发战略计划更新的结构此更新的国家AI研发战略计划包含2016版的原始文本,包括以下关于R&D战略的部分(小编辑除外)和前七个战略的原始2016年措辞.
对于每项战略,2019年国家研发战略计划的更新在最初七项战略的顶部以阴影框提供;这些突出了战略的更新要求和/或新的重点领域.
阴影框下面的文本最初出现在2016年国家AI研发战略计划中,提供了今天仍然重要的观察和背景(请注意,在此期间,一些原始细节可能已经过时).
此外,如前所述,2019年更新中增加了一项新的第八项战略,即扩大人工智能研发中的公共私营伙伴关系.
国家人工智能研发战略计划:2019–9–人工智能研发战略本AI人力资源研发战略计划中概述的研究重点集中在行业不可能单独解决的领域,因此,最有可能从联邦投资中获益的领域.
这些优先事项贯穿整个AI,包括AI感知子领域,自动推理/规划,认知系统,机器学习,自然语言处理,机器人和相关领域的共同需求.
由于人工智能的广度,这些优先事项涵盖整个领域,而不是仅关注每个子领域特有的个别研究挑战.
为了实施该计划,应制定详细的路线图,以解决与计划一致的能力差距.
战略1中概述的最重要的联邦研究重点之一是持续进行人工智能的长期研究,以推动发现和洞察力.
许多投资受到美国联邦政府的高风险,高回报9基础研究已经带来了我们今天依赖的革命性技术进步,包括互联网,GPS,智能手机语音识别,心脏监测器,太阳能电池板,先进电池,癌症治疗等等.
人工智能的承诺几乎涉及社会的各个方面,并具有显着的积极的社会和经济效益的潜力.
因此,为了在这一领域保持世界领先地位,美国必须将其投资重点放在高优先级的基础和长期人工智能研究上.
许多人工智能技术将与人类一起工作,从而在如何最好地创建以直观和有用的方式与人合作的人工智能系统方面面临重大挑战.
10人类和人工智能系统之间的墙壁正逐渐开始腐蚀,人工智能系统增强并增强了人类的能力.
如战略2所述,需要进行基础研究,以制定人与人之间互动和协作的有效方法.
人工智能的进步为社会带来了许多积极的好处,并提高了美国的国家竞争力.
11然而,与大多数变革性技术一样,人工智能在一些领域存在一些社会风险,从就业和经济到安全,道德和法律问题.
因此,随着人工智能科学和技术的发展,联邦政府还必须投资研究,以更好地了解人工智能对所有这些领域的影响,并通过开发符合道德,法律和社会目标的人工智能系统来解决这些问题.
,如战略3所述.
当前AI技术的一个关键缺口是缺乏确保AI系统安全性和可预测性能的方法.
由于这些系统具有不同寻常的复杂性和不断发展的特性,因此确保AI系统的安全性是一项挑战.
一些研究重点解决了这一安全挑战.
首先,策略4强调需要可由用户信任的可解释和透明的系统,以用户可接受的方式执行,并且可以保证充当用户的意图.
人工智能系统的潜在能力和复杂性,加上与人类用户和环境的可能互动,使得投资于提高人工智能技术安全性和控制力的研究至关重要.
战略5呼吁联邦政府投资共享公共数据集进行人工智能培训和测试,以推进人工智能研究的进展,并能够更有效地比较替代解决方案.
战略6讨论了标准和基准如何能够集中研发来定义进度,缩小差距,并针对特定问题和挑战推动创新解决方案.
标准和基准是9"高风险,高回报"的研究指的是具有智力挑战性但有可能对研究领域产生深刻积极的变革性影响的有远见的研究.
10见2016年人工智能百年研究报告,重点关注人工智能在2030年的预期用途和影响;https://ai100.
stanford.
edu/2016-report.
11J.
弗曼,"这次不同吗人工智能的机遇与挑战,"经济顾问委员会评论,纽约大学:AI现在研讨会,2016年7月7日.
国家人工智能研发战略计划:2019–10–对于测量和评估AI系统以及确保AI技术满足功能和互操作性的关键目标至关重要.
最后,人工智能技术在社会各个领域日益普及,给人工智能研发专家带来了新的压力.
核心人工智能科学家和工程师的机会比比皆是,他们对技术有深刻的理解,可以为推动该领域知识的界限产生新的想法.
国家应采取行动,确保有足够的人工智能人才管道.
战略7解决了这一挑战.
图1(在2019版本的计划中更新)提供了此AI研发战略计划的整体组织的图形说明.
在底部的方框中是横切的,底层基础,影响所有AI系统的发展;战略3-7和新战略8中描述了这些基础.
下一层更高(中间一排方框)包括推进AI所需的许多研究领域.
策略1-2概述了这些研发领域(包括使用灵感的基础研究).
12图中顶行的方框是预计将受益于AI进步的应用程序示例.
人工智能研发战略计划的这些组成部分共同确定了联邦投资的高级框架,可以在该领域产生有影响的进展并带来积极的社会效益.
图1.
人工智能研发战略计划的组织(2019年更新,包括战略8).
横切R&D基础的组合(在下排)对于所有AI研究都很重要.
许多人工智能研发领域(在中间一行)可以建立在这些横切基础上,以影响各种社会应用(在第一行).
括号内的数字表示该计划中进一步发展每个主题的战略数量.
这些策略的排序并不表示重要性.
12在整个文件中,"基础研究"包括纯粹的基础研究和基于使用的基础研究-所谓的巴斯德象限,由唐纳德斯托克斯在他1997年的同名书中定义-指的是用于基础研究的基础研究社会在想.
例如,NIH对IT的基本投资通常被称为使用启发式基础研究.
国家人工智能研发战略计划:2019–11–战略1:对人工智能研究进行长期投资2019更新持续对基础AI研究进行长期投资自2016年国家AI研发战略计划发布以来,强大的新功能,主要是ML应用程序,以明确定义的任务,不断涌现.
这些能力已经在各种应用中表现出了影响,例如对基因序列进行分类,20,21管理有限的无线频谱资源,22解释医学图像,23和癌症分级.
24这些快速进步需要数十年的研究才能使技术和应用成熟.
25为了在ML中保持这一进展以实现AI的其他领域的进步,并努力实现通用AI的长期目标,联邦政府必须继续促进ML和AI的长期,基础研究.
这项研究将产生转型技术,并反过来在社会各阶层实现突破.
该领域当前的许多进展一直是专门的,定义明确的任务,通常由统计ML驱动,例如分类,识别和回归(即"窄AI系统").
调查结果13https://www.
nsf.
gov/funding/pgm_summ.
jsppims_id=505640&org=NSF14https://www.
nsf.
gov/pubs/2019/nsf19051/nsf19051.
jsp15https://www.
nsf.
gov/pubs/2019/nsf19018/nsf19018.
jsp16https://www.
nsf.
gov/funding/pgm_summ.
jsppims_id=50565117https://www.
nsf.
gov/cise/harnessingdata/18https://www.
darpa.
mil/work-with-us/ai-next-campaign19https://datascience.
nih.
gov/strategicplan20https://ai.
googleblog.
com/2017/12/deepvariant-highly-accurate-genomes.
html21https://irp.
nih.
gov/catalyst/v26i4/machine-learning22https://www.
spectrumcollaborationchallenge.
com/23https://news-medical.
net/news/20190417/Workshop-explores-the-future-of-artificial-intelligence-in-medical-imaging.
aspx24https://www.
nature.
com/articles/nature2105625https://www.
nitrd.
gov/rfi/ai/2018/AI-RFI-Response-2018-Yolanda-Gil-AAAI.
pdf科学14-与行业.
15,16此外,NSF正在利用数据革命的大创意17支持对数据科学基础的研究,这将成为未来ML和AI系统的驱动力.
DARPA于2018年9月宣布对新的和现有的计划进行多年投资,称为"AINext"活动.
18主要活动领域包括提高AI系统的稳健性和可靠性;加强ML/AI技术的安全性和弹性;降低功耗,数据和性能低效;并开创了下一代AI算法和应用,例如可解释性和常识推理.
NIH数据科学战略计划192018年9月旨在促进生物医学研究界获得数据科学技术和ML/AI能力,以实现数据驱动的医疗保健研究.
以及USDA-NIFA对农业的AI毫升13自2016年国家航空与战略计划发布以来,一些机构已启动战略1的人工智能研发计划:NSF继续资助人工智能的基础研究,包括ML,推理和表示,计算机视觉,计算神经科学,语音和语言,机器人和多智能体系统.
NSF与其他机构合作推出了新的联合融资机会-尤其是DARPA在高性能,高能效硬件领域的实时性长期,基础的人工智能研究:近期的机构研发计划国家人工智能研发战略计划:2019–12–该领域已经指出,需要对基础研究进行长期投资,以继续在ML的这些进步基础上再接再厉.
此外,需要并行持续努力才能充分实现"通用AI"系统的愿景,该系统在广泛的认知领域中展现出人类智能的灵活性和多功能性.
26,27,28,29需要强调开发进一步的ML能力,以交互式和持续学习,感知和注意之间的联系,以及将学习模型纳入综合推理架构.
30除了ML之外,人工智能的其他核心领域也需要进行批判性研究,包括常识推理和问题解决,概率推理,组合优化,知识表示,计划和调度,自然语言处理,决策制定和人机交互.
这些领域的进步将反过来实现协作机器人以及共享和完全自治的系统(参见策略2).
理解人类智能的巨大挑战需要对共享资源和基础设施进行大量投资.
25对ML和AI驱动程序的基础投资也存在广泛的共识,包括数据来源和质量,新颖的软件和硬件范例和平台,以及AI系统的安全性.
31,32例如,随着人工智能软件在日常生活和经济的各个方面执行越来越复杂的功能,现有的软件开发范例将需要发展以满足软件生产力,质量和可持续性要求.
最近的联邦投资优先考虑了基础ML和AI研究的这些领域(见边栏),以及ML和AI在众多应用领域的应用,包括国防,安全,能源,交通,卫生,农业和电信.
最终,人工智能技术对于解决一系列长期挑战至关重要,例如构建先进的医疗保健系统,强大的智能交通系统以及弹性能源和电信网络.
为了使AI应用程序变得普及,它们必须是可解释和可理解的(参见策略3).
这些挑战对于促进人与人之间的人际关系协作尤为突出(见战略2).
如今,理解和分析AI系统决策并测量其准确性,可靠性和可重复性的能力是有限的.
需要持续的研发投资来提高对人工智能系统的信任,以确保满足社会需求并充分满足稳健性,公平性,可解释性和安全性的要求.
对人工智能研发的长期承诺对于继续和扩展当前的技术进步至关重要,并且更广泛地确保人工智能丰富了人类的经验.
事实上,2019年"维持美国人工智能领导力行政命令"指出:执行或资助研发(AI研发机构)的执行机构负责人应将AI视为代理机构的研发优先事项,视各自机构的任务而定.
这些机构的负责人在制定预算提案和规划时应考虑到这一优先事项.
在2020财年和未来几年使用资金.
这些机构的负责人还应考虑采取适当的行政行动,以提高对2019年人工智能的关注.
26https://ai100.
stanford.
edu/sites/g/files/sbiybj9861/f/ai100report10032016fnl_singles.
pdf27http://cdn.
AdQue.
org/2018/Ai%20%索引202018%,2020%20RePort.
PDF28https://cra.
org/ccc/visioning/visioning-activities/2018-activities/artificial-intelligence-roadmap/29https://www.
microsoft.
com/en-us/research/research-area/artificial-intelligence/30https://cra.
org/ccc/events/artificial-intelligence-roadmap-workshop-3-learning-and-robotics/31https://cra.
org/ccc/wp-content/uploads/sites/2/2016/04/AI-for-Social-Good-Workshop-Report.
pdf32https://openai.
com/blog/ai-and-compute/国家人工智能研发战略计划:2019–13–在具有潜在长期收益的领域,需要进行人工智能研究投资.
虽然长期研究的一个重要组成部分是具有可预测结果的渐进式研究,但对高风险研究的长期持续投资可以带来高回报的回报.
这些回报可以在5年,10年或更长时间内看到.
2012年国家研究委员会的一份报告强调了联邦投资在长期研究中的关键作用,并指出"长期不可预测的潜伏期-需要稳定的工作和资金-在初始勘探和商业部署之间".
33它进一步指出,"从第一个概念到成功市场的时间通常是在几十年内测量的.
"有充分记录的持续基础研究工作的例子包括万维网和深度学习.
在这两种情况下,基本的基础始于20世纪60年代;只有经过30多年的持续研究努力,这些想法才能体现到今天在许多类别的人工智能中见证的变革性技术.
以下小节重点介绍其中一些方面.
战略2至6讨论了其他类别的重要人工智能研究.
推进以数据为中心的知识发现方法如2016年联邦大数据研究与发展战略计划中所述,34需要许多基本的新工具和技术来实现智能数据理解和知识发现.
在开发更先进的机器学习算法时需要进一步的进展,这些算法可以识别隐藏在大数据中的所有有用信息.
许多开放式研究问题围绕着数据的创建和使用,包括其对AI系统培训的准确性和适当性.
在处理大量数据时,数据的准确性尤其具有挑战性,使人们难以从中评估和提取知识.
虽然许多研究通过数据质量保证方法处理数据清理和知识发现的准确性,但还需要进一步研究以提高数据清理技术的效率,创建发现数据不一致和异常的方法,并制定方法.
纳入人类反馈.
研究人员需要探索新方法,以便同时挖掘数据和相关元数据.
许多AI应用程序本质上是跨学科的,并且使用异构数据.
需要进一步研究多模态机器学习以使得能够从各种不同类型的数据(例如,离散的,连续的,文本的,空间的,时间的,时空的,图形的)中发现知识.
AI调查人员必须确定培训所需的数据量,并妥善解决大规模和长尾数据需求.
除了纯粹的统计方法之外,他们还必须确定如何识别和处理罕见事件;利用知识来源(即解释世界的任何类型的信息,例如重力法或社会规范的知识)以及数据来源,在学习过程中整合模型和本体;当大数据源可能不可用时,用很少的数据获得有效的学习成绩.
增强AI系统的感知能力感知是智能系统进入世界的窗口.
感知始于(可能是分布式的)传感器数据,其具有不同的形式和形式,例如系统本身的状态或有关环境的信息.
传感器数据经过处理和融合,通常与先验知识和模型一起,提取与AI系统任务相关的信息,如33国家研究委员会计算机科学电信委员会,信息技术的持续创新(国家科学院出版社,华盛顿特区,2012年),11;https://doi.
org/10.
17226/13427.
34https://www.
nitrd.
gov/PUBS/bigdatardstrategicplan.
pdf国家人工智能研发战略计划:2019–14–几何特征,属性,位置和速度.
来自感知的综合数据形成态势感知,为人工智能系统提供全面的知识和有效安全地规划和执行任务所必需的世界状况模型.
AI系统将极大地受益于硬件和算法的进步,以实现更强大和可靠的感知.
传感器必须能够以更高的分辨率和实时捕获更远距离的数据.
感知系统需要能够整合来自各种传感器和其他来源(包括计算云)的数据,以确定AI系统当前感知的内容并允许预测未来状态.
物体的检测,分类,识别和识别仍然具有挑战性,尤其是在杂乱和动态的条件下.
此外,通过使用传感器和算法的适当组合,必须大大提高对人类的感知,以便AI系统可以更有效地与人们一起工作.
10需要在整个感知过程中计算和传播不确定性的框架,以量化AI系统在其态势感知中的置信水平并提高准确性.
了解AI的理论能力和局限性虽然许多AI算法的最终目标是通过类似人类的解决方案解决开放性挑战,但我们并未充分了解AI的理论能力和局限性以及此类人类解决方案甚至可能实现的程度.
AI算法.
需要理论工作来更好地理解为什么人工智能技术-特别是机器学习-在实践中经常运作良好.
虽然不同的学科(包括数学,控制科学和计算机科学)正在研究这个问题,但该领域目前缺乏统一的理论模型或框架来理解AI系统的性能.
需要对计算可解性进行额外的研究,这是对AI算法在理论上能够解决的问题类别的理解,同样也是对它们无法解决的问题的理解.
必须在现有硬件的环境中开发这种理解,以便了解硬件如何影响这些算法的性能.
了解哪些问题在理论上无法解决,可以使研究人员针对这些问题制定近似解决方案,甚至开辟新的AI系统硬件研究新路线.
例如,当发明于20世纪60年代时,人工神经网络(ANNs)只能用于解决非常简单的问题.
在进行并行化等硬件改进后,使用人工神经网络来解决复杂问题变得可行,并且调整算法以利用新硬件.
这些发展是实现当今深度学习取得重大进展的关键因素.
开展通用人工智能研究AI方法可以分为"窄AI"和"一般AI".
窄AI系统在专门的,定义明确的域中执行单独的任务,例如语音识别,图像识别和翻译.
几个最近,高度可见,狭窄的AI系统,包括IBMWatson和DeepMind的AlphaGo,已经取得了重大成就.
35,36事实上,这些特殊的系统被称为"超人",因为它们的表现超过了Jeopardy中最优秀的人类玩家!
和Go分别.
但是这些系统举例说明了狭义的AI,因为它们只能应用于专门设计的任务.
在更广泛的问题上使用这些系统需要大量的重新设计工作.
相比之下,一般人工智能的长期目标是创建系统352011年,IBMWatson击败了两名被认为是Jeopardy中最佳人类玩家的玩家!
游戏.
362016年,AlphaGo击败了Go,LeeSe-dol的卫冕世界冠军.
值得注意的是,AlphaGo结合了深度学习和蒙特卡罗搜索-一种在20世纪80年代发展起来的方法-它本身建立在20世纪40年代发现的概率方法上.
国家人工智能研发战略计划:2019–15–在广泛的认知领域展示人类智能的灵活性和多功能性,包括学习,语言,感知,推理,创造力和计划.
广泛的学习能力将为一般人工智能系统提供将知识从一个领域转移到另一个领域并以交互方式从经验和人类中学习的能力.
自人工智能出现以来,人工智能一直是研究人员的野心,但目前的系统还远未实现这一目标.
目前正在探索狭义和一般人工智能之间的关系;可以应用一个课程来改进另一个课程,反之亦然.
虽然没有普遍的共识,但大多数人工智能研究人员认为,一般人工智能仍然需要几十年的时间,需要长期,持续的研究努力来实现它.
开发可扩展的AI系统AI系统的组和网络可以协调或自主协作以执行单个AI系统不可能完成的任务,并且还可以包括与团队一起工作或领导团队的人.
这种多AI系统的开发和使用在这些系统的规划,协调,控制和可扩展性方面产生了重大的研究挑战.
多AI系统的规划技术必须足够快,以便实时操作并适应环境的变化.
它们应该以流畅的方式适应可用通信带宽或系统退化和故障的变化.
以前的许多努力都集中在集中规划和协调技术上;但是,这些方法受到单点故障的影响,例如计划员的丢失或与计划员的通信链接的丢失.
分布式规划和控制技术在算法上难以实现,并且通常效率低且不完整,但可能对单点故障提供更强的鲁棒性.
未来的研究必须发现更有效,更强大,可扩展的技术,用于多个AI系统和人类团队的规划,控制和协作.
促进人类AI的研究获得类似人类的AI要求系统以人们可以理解的方式解释自己.
这将产生新一代智能系统,例如智能辅导系统和智能助理,可有效地协助人们执行任务.
然而,当前AI算法的工作方式与人们学习和执行任务的方式之间存在巨大差距.
人们只能从几个例子中学习,或通过接受正式指导和/或"提示"来执行任务,或通过观察执行这些任务的其他人.
医学院采用这种方法,例如,医学生通过观察一位执行复杂医疗程序的老医生来学习.
即使在世界冠军Go游戏这样的高性能任务中,一个大师级玩家也只能玩几千个游戏来训练他/她自己.
相比之下,人类需要数百年才能玩出训练AlphaGo所需的游戏数量.
关于实现人类AI的新方法的更多基础研究将使这些系统更接近这一目标.
开发更强大,更可靠的机器人在过去十年中,机器人技术的重大进步正在导致多种应用的潜在影响,包括制造,物流,医药,医疗保健,国防和国家安全,农业和消费品.
虽然历史上设想机器人用于静态工业环境,但最近的进展涉及机器人和人类之间的密切合作.
现在,机器人技术在补充,增强,增强或模拟人体物理能力或人类智能方面具有前景.
然而,科学家需要使这些机器人系统更有能力,更可靠,更易于使用.
国家人工智能研发战略计划:2019–16–研究人员需要更好地了解机器人感知,从各种传感器中提取信息,为机器人提供实时态势感知.
在认知和推理方面需要取得进展,以使机器人能够更好地理解物理世界并与之互动.
改进的适应和学习能力将使机器人能够概括他们的技能,对他们当前的表现进行自我评估,并从人类教师那里学习一些物理运动.
移动性和操纵性是进一步研究的领域,因此机器人可以穿越崎岖不确定的地形并灵活处理各种物体.
机器人需要学会以无缝方式团结在一起,并以可靠和可预测的方式与人类合作.
推进硬件以改进AI尽管人工智能研究通常与软件的进步有关,但人工智能系统的性能在很大程度上取决于其运行的硬件.
目前深度机器学习的复兴与基于GPU的硬件技术及其改进的内存的进步直接相关,37输入/输出,时钟速度,并行度和能效.
开发针对AI算法优化的硬件将实现比GPU更高的性能水平.
一个例子是"神经形态"处理器,其受到大脑组织的松散启发,并且在某些情况下,针对神经网络的操作进行了优化.
38硬件的进步还可以提高高度数据密集的AI方法的性能.
需要进一步研究在整个分布式系统中以受控方式打开和关闭数据管道的方法.
还需要继续研究以允许机器学习算法有效地从高速数据中学习,包括同时从多个数据流水线中学习的分布式机器学习算法.
更先进的基于机器学习的反馈方法将允许AI系统智能地对来自大规模模拟,实验仪器和分布式传感器系统(如智能建筑和物联网(IoT))的数据进行采样或优先排序.
这些方法可能需要动态I/O决策,其中基于重要性或重要性实时选择存储数据,而不是简单地以固定频率存储数据.
创建AI以改进硬件虽然改进的硬件可以带来更强大的AI系统,但AI系统也可以提高硬件的性能.
39这种互易性将导致硬件性能的进一步提高,因为计算的物理限制需要新颖的硬件设计方法.
40基于AI的方法对于改进高性能计算(HPC)系统的运行尤其重要.
这种系统消耗大量能量.
AI用于预测HPC性能和资源使用情况,并做出可提高效率的在线优化决策;更先进的AI技术可以进一步提高系统性能.
AI也可以用来创建37GPU代表图形处理单元,它是一个功耗和成本效益的处理器,包含数百个处理内核;这种设计使其特别适用于固有的并行应用,包括大多数AI系统.
38神经形态计算是指硬件学习,适应和物理重新配置的能力,从生物学或神经科学中获取灵感.
39M.
Milano和L.
Benini,"HPC系统中工作功耗的预测建模","高性能计算会议论文集:31圣国际会议,ISC高性能2016"(SpringerVol.
9697,2016).
40这些计算的物理限制被称为Dennard缩放,并导致高的片上功率密度和称为"暗硅"的现象,其中需要关闭芯片的不同部分以限制温度并确保数据完整性.
国家人工智能研发战略计划:2019–17–自重构HPC系统,可在发生系统故障时处理,无需人工干预.
41改进的AI算法可以通过减少处理器和内存之间的数据移动来提高多核系统的性能-这是运行速度比当今超级计算机快10倍的亿亿次级计算系统的主要障碍.
42实际上,HPC系统中的执行配置永远不会相同,并且不同的应用程序同时执行,每个不同的软件代码的状态在时间上独立发展.
人工智能算法需要设计为在线和大规模运行HPC系统.
41A.
Cocaa-Fernández,J.
Ranilla和L.
Sánchez,"通过参数学习和混合遗传模糊系统建模在HPC集群中计算节点插槽的节能分配","超级计算杂志"71(2015):1163-1174.
42Exascale计算系统每秒至少可以实现10亿次计算.
国家人工智能研发战略计划:2019–18–战略2:开发有效的人工智能协作方法2019更新开发人工智能系统,补充和增强人的能力,更加关注工作的未来自2016年国家AI研发战略计划发布以来,国家对人工智能合作的兴趣不断增加.
当AI系统补充和增强人类能力时,人类和AI成为一系列共享到完全自治场景的合作伙伴.
特别是,人工智能合作在工作的未来背景下既是挑战又是机遇.
在过去三年中,新成立的以及长期的会议,研讨会和工作组广泛地优先考虑人与人之间的合作.
例如,人类计算和众包会议已经从一个研讨会发展成为一个重要的国际会议,该会议促进了人工智能与人机交互(HCI)交叉的研究.
452018年,人工智能促进协会选择人工智能合作作为其年度会议的新兴主题.
462019年5月,CHI举办了规模最大的人机交互会议,其中包括"弥合人工智能和人机交互之间差距"的研讨会.
47"人机交互"杂志于2019年3月发出电话,要求就"统一人机交互和人工智能"这一特刊提交意见书.
4843https://www.
nsf.
gov/eng/futureofwork.
jsp44DOE研讨会报告,科学机器学习的基础研究需求:人工智能的核心技术:https://www.
osti.
gov/biblio/1478744.
45欢迎来到HCOMP2019:https://www.
humancomputation.
com/.
46AAAI-18新兴主题人工智能协作:http://www.
aaai.
org/Conferences/AAAI/2018/aaai18emergingcall.
php.
47人类在哪里缩小人工智能和人机交互之间的差距:CHI2019研讨会:https://michae.
lv/ai-hci-workshop/.
48呼叫:"统一人机交互和人工智能"人机交互问题:https://ispr.
info/2019/02/20/call-unifying-human-computer-interaction-and-artificial-intelligence-issue-of-human-计算机交互/.
自2016年国家人工智能研发战略计划发布以来,一些机构已开始为战略2做出努力:NSF在人类技术前沿的未来工作43BigIdea正在支持社会技术研究,使智能技术与人类协同合作,实现对员工的广泛参与,并在一系列工作环境中改善社会,经济和环境效益.
NOAA(国家海洋和大气管理局)正在推进飓风,龙卷风和其他恶劣天气预报的人工智能合作,其中人类预报员和人工智能系统共同努力改善恶劣天气警报的产生,并确定作为极端前兆的独特模式事件.
人类预报员有时被称为"人类在环路之上",监督人工智能系统的预测并指导结果.
NIH正在进行自然语言处理研究,该数据库基于从国家医学图书馆维护的所有MEDLINE引文中提取的9630万个事实数据库.
2019年美国能源部关于科学机器学习的研讨会报告确定了优先研究方向,主要科学用例以及人类与人工智能合作将改变科学研究方式的新趋势.
44人工智能协作:最近的代理研发计划国家人工智能研发战略计划:2019–19–在工作的背景下,会议开始探索人类,机器及其伙伴关系的作用,如麻省理工学院的计算机科学和人工智能实验室(CSAIL)以及推出年度人工智能和未来的数字经济倡议工作大会.
49,50作为美国人工智能研究20年社区路线图的一部分,51在2019年,计算社区联盟(CCC)举办了一个研讨会,重点关注人与人工智能系统之间的有意义的互动.
52此外,CCC在2017-2018年期间运营人类技术前沿工作组,专注于技术的潜力,以提高人员绩效,包括但不限于工作场所,教室和医疗保健系统.
532016年国家人工智能研发战略计划中的交叉战略原则,"人工智能系统的适当信任需要可解释性,特别是随着人工智能在规模和复杂性方面的增长,"在人工智能合作的背景下,已经看到了研发行动呼吁.
一些专业协会和机构已将这一原则确定为优先领域(见附文).
该研究领域反映了战略2和3的交叉,因为可解释性,公平性和透明性是人工智能系统与人类有效合作的关键原则.
同样,理解和设计人工智能伦理和价值调整系统的挑战仍然是一个开放的研究领域.
与此同时,私营部门已经采取了有效的人工智能合作原则.
54,55由于联邦机构在过去三年中根据任务目标增加了人工智能投资,因此他们共同强调人机认知,自主权和代理,例如决策支持,风险建模,态势感知和可信机器智能(见边栏).
通过此类研发投资,研究伙伴关系正在越来越多的轴上发展,将计算科学家聚集在一起;行为,认知和心理科学家;和其他领域的科学家和工程师.
学术研究人员和工作场所内外人工智能系统用户之间已经形成了新的合作关系.
展望未来,联邦机构必须继续在开放世界中促进人工智能研发,以促进人工智能系统的设计,这些系统包含并适应用户的情况和目标,以便人工智能系统和用户能够在预期和未预料到的情况下协同工作.
虽然完全自主的AI系统在某些应用领域(例如,水下或深空探测)中非常重要,但许多其他应用领域(例如,灾难恢复和医疗诊断)通过人类和AI系统的组合最有效地解决.
实现应用目标.
这种协作互动利用了人类和AI系统的互补性.
虽然人工智能协作的有效方法已经存在,但其中大多数都是"点解决方案",只能在特定环境中使用特定平台实现特定目标.
为每个可能的应用程序实例生成点解决方案无法扩展;因此,需要做更多工作才能超越这些点解决方案49https://futureofwork.
csail.
mit.
edu/.
50AI和2019年工作创新峰会的未来:https://analyticsevent.
com/.
51https://cra.
org/ccc/wp-content/uploads/sites/2/2019/03/AI_Roadmap_Exec_Summary-FINAL-.
pdf52人工智能路线图研讨会2-互动:https://cra.
org/ccc/events/artificial-intelligence-路线图车间-2-相互作用/.
53https://cra.
org/ccc/human-technology-frontier/54https://www.
microsoft.
com/en-us/research/uploads/prod/2019/01/Guidelines-for-Human-AI-Interaction-camera-ready.
pdf55https://www.
partnershiponai.
org/about/#our-work国家人工智能研发战略计划:2019–20–人工智能协作的一般方法.
必须在设计适用于所有类型问题的一般系统,需要较少的人力建设和更大的应用程序之间切换设施之间进行权衡,而不是构建大量特定于问题的系统,这些系统可以更有效地解决每个问题.
未来的应用在人类和人工智能系统之间的功能角色划分,人类与人工智能系统之间的相互作用的性质,人类和其他人工智能系统协同工作的数量以及人类和人工智能系统如何沟通和分享态势感知方面会有很大差异.
.
人与AI系统之间的功能角色划分通常属于以下类别之一:1.
AI与人类一起执行功能:AI系统执行支持人类决策者的外围任务.
例如,AI可以帮助人们进行工作记忆,短期或长期记忆检索以及预测任务.
2.
当人类遇到高认知过载时,AI执行功能:AI系统执行复杂的监控功能(例如飞机中的近地警告系统),决策制定以及人类需要帮助时的自动医疗诊断.
3.
AI代替人类执行功能:AI系统执行人类能力非常有限的任务,例如复杂的数学运算,有争议的操作环境中动态系统的控制指导,有害或有毒环境中自动化系统的控制方面,在系统响应非常迅速的情况下(例如,在核反应堆控制室).
实现人与人工智能系统之间的有效互动需要额外的研发,以确保系统设计不会导致过度的复杂性,不确定性或推翻.
通过培训和经验可以增加人类对人工智能系统的熟悉程度,以确保人类对人工智能系统的能力以及人工智能系统能够做什么和不能做什么有很好的理解.
为解决这些问题,应在这些系统的设计和开发中使用某些以人为中心的自动化原则:561.
采用直观,用户友好的人工智能系统界面,控件和显示设计.
2.
让操作员知情.
显示关键信息,AI系统的状态以及这些状态的变化.
3.
让操作员接受培训.
参与一般知识,技能和能力(KSA)的复训,以及AI系统采用的算法和逻辑培训以及系统的预期故障模式.
4.
使自动化灵活.
部署AI系统应被视为希望决定是否要使用它们的操作员的设计选项.
同样重要的是自适应AI系统的设计和部署,可用于在过度工作负荷或疲劳期间为人类操作员提供支持.
57,58在创建与人类有效合作的系统时,研究人员面临许多基本挑战.
以下小节概述了其中一些重要挑战.
56C.
Wickens和JGHollands,"注意力,分时和工作量.
"在工程,心理学和人类表现(伦敦:PearsonPLC,1999),439-479.
57https://www.
nasa.
gov/mission_pages/SOFIA/index.
html58https://cloud1.
arc.
nasa.
gov/intex-na/国家人工智能研发战略计划:2019–21–寻求人类感知AI的新算法多年来,AI算法已经能够解决日益复杂的问题.
然而,这些算法的能力与人类对这些系统的可用性之间存在差距.
需要人性化智能系统,可以直观地与用户进行交互,实现无缝的机器人与人之间的协作.
直观的交互包括浅层交互,例如当用户丢弃系统推荐的选项时;基于模型的方法,考虑到用户过去的行为;甚至是基于准确的人类认知模型的用户意图的深度模型.
必须开发中断模型,允许智能系统仅在必要和适当时中断人类.
智能系统还应具有增强人类认知的能力,知道在用户需要时检索哪些信息,即使他们没有明确地提示系统获取该信息.
未来的智能系统必须能够考虑人类的社会规范并采取相应的行动.
智能系统如果具有某种程度的情商,就能更有效地与人类合作,以便他们能够识别用户的情绪并做出适当的反应.
另一个研究目标是超越一个人和一个机器的交互,转向"系统系统",即由多个机器与多个人交互组成的团队.
人工智能系统交互具有广泛的目标.
人工智能系统需要能够代表多个目标,他们可以采取的行动来实现这些目标,对这些行动的约束以及其他因素,以及轻松适应目标的修改.
此外,人类和人工智能系统必须有共同的目标,并且能够相互理解它们以及它们当前状态的相关方面.
需要进一步研究以概括人类AI系统的这些方面,以开发需要较少人工工程的系统.
开发用于人体增强的AI技术虽然人工智能研究的大部分重点都放在与执行狭隘任务的人匹配或表现优异的算法上,但需要开展更多工作来开发能够跨越多个领域增强人类能力的系统.
人体增强研究包括在固定设备(例如计算机)上工作的算法;可穿戴设备(如智能眼镜);植入设备(如大脑界面);在特定的用户环境中(例如特别定制的手术室).
例如,基于从多个设备组合的数据读数,增强的人类意识可以使医疗助理指出医疗过程中的错误.
其他系统可以通过帮助用户回忆过去适用于用户当前情况的经验来增强人类认知.
人类与人工智能系统之间的另一种协作方式是积极学习智能数据.
在主动学习中,从领域专家寻求输入,并且仅在学习算法不确定时才对数据进行学习.
这是减少首先需要生成的训练数据量或需要学习的量的重要技术.
主动学习也是获得领域专家意见和增加学习算法信任的关键方法.
到目前为止,主动学习仅用于监督学习;需要进一步研究将主动学习纳入无监督学习(例如,聚类,异常检测)和强化学习.
59概率网络允许以先验概率分布的形式包括领域知识.
无论是以数学模型,文本还是其他形式,都必须寻求允许机器学习算法合并领域知识的一般方法.
59虽然有监督的学习需要人类提供真实的答案,强化学习和无监督学习却没有.
国家人工智能研发战略计划:2019–22–开发可视化和人工智能接口技术更好的可视化和用户界面是需要更多开发的其他领域,以帮助人们理解来自各种来源的大量现代数据集和信息.
可视化和用户界面必须以人类可理解的方式清楚地呈现日益复杂的数据和从中获取的信息.
提供实时结果对于安全关键操作非常重要,并且可以通过增加计算能力和连接系统来实现.
在这些类型的情况下,用户需要可视化和用户界面,可以快速传达正确的信息以进行实时响应.
人工智能协作可以应用于各种环境,并且存在通信限制.
在某些领域,人工智能通信延迟很低,通信快速可靠.
在其他领域(例如,美国国家航空航天局部署漫游者精神和机遇到火星),人类与人工智能系统之间的远程通信具有非常高的潜伏期(例如,地球和火星之间的往返时间为5-20分钟),因此需要部署的平台在很大程度上自主运行,只有高层战略目标传达给平台.
这些通信要求和约束是用户界面研发的重要考虑因素.
开发更有效的语言处理系统人们通过口头和书面语言与人工智能系统进行交互一直是人工智能研究人员的目标.
虽然取得了重大进展,但在人类可以与人工智能系统进行有效沟通之前,语言处理必须解决相当大的开放性研究挑战,就像对待其他人类一样.
语言处理的最新进展被认为是数据驱动的机器学习方法的使用,这导致了成功的系统,例如,成功地在安静的环境中实时识别流利的英语语音.
然而,这些成就只是实现长期目标的第一步.
当前的系统无法应对现实世界的挑战,例如嘈杂环境中的语音,重音口音,儿童语音,语音障碍和手语语音.
还需要开发能够与人类进行实时对话的语言处理系统.
这样的系统需要推断其人类对话者的目标和意图,使用适当的登记,风格和修辞来处理这种情况,并在对话误解的情况下采用修复策略.
需要进一步研究开发更容易概括不同语言的系统.
此外,需要进一步研究以语言处理系统易于访问的形式获取有用的结构化领域知识.
还需要在许多其他领域中进行语言处理,以使人与AI系统之间的交互更加自然和直观.
必须为口头和书面语言的模式构建强大的计算模型,这些模式为情绪状态,情感和立场提供证据,并用于确定言语和文本中隐含的信息.
对于在物理世界中运行的AI系统,例如在机器人技术中,需要新的语言处理技术来为环境环境中的语言奠定基础.
最后,由于人们在在线交互中进行交流的方式可能与语音交互完全不同,因此必须完善在这些环境中使用的语言模型,以便社交AI系统可以更有效地与人交互.
国家人工智能研发战略计划:2019–23–策略3:理解并解决人工智能的道德,法律和社会影响2019更新解决人工智能中的道德,法律和社会问题自2016年国家人工智能研发战略计划发布以来,针对人工智能系统开发和部署的道德,法律和社会影响的研发活动有所增加.
人们越来越认识到,人工智能系统必须"值得信赖",人工智能可以改变社会和经济生活的许多领域,包括就业,医疗保健和制造业.
经济合作与发展组织(OECD)等国际组织63和G7创新部长64鼓励研发增加对人工智能的信任和采用.
2016年国家AI研发战略计划在确定隐私研究主题方面具有先见之明;通过设计提高AI系统的公平性,透明度和责任性;并设计道德AI的架构.
致力于ML和AI系统的公平性,问责制和透明度的研究会议蓬勃发展.
65联邦机构已经回应了各种新的研究计划和会议,重点关注这些关键领域(见附文).
60https://www.
darpa.
mil/program/explainable-artificial-intelligence61"2018年国防部人工智能战略总结":https://media.
defense.
gov/2019/Feb/12/2002088963/-1/-1/1/SUMMARY-OF-DOD-AI-STRATEGY.
PDF.
62https://www.
nsf.
gov/pubs/2019/nsf19571/nsf19571.
htm63"经合组织关于人工智能的倡议":http://www.
oecd.
org/going-digital/ai/oecd-initiatives-on-ai.
htm.
64"G7创新部长关于人工智能的声明":http://www.
g8.
utoronto.
ca/employment/2018-labour-annex-b-en.
html.
65http://www.
fatml.
org/;https://fatconference.
org/;http://www.
aies-conference.
com/自2016年国家AI研发战略计划发布以来,一些机构已启动战略3的AI研发计划:DARPA的可解释AI(XAI)计划60旨在创建一套ML技术,生成更多可解释的AI系统,同时保持高水平的学习性能(预测准确性).
XAI还将使人类用户能够理解,适当地信任并有效地管理新一代AI系统.
更一般地,国防部致力于"引领军事道德和人工智能安全",作为指导其加速采用人工智能系统的战略方法中列出的五项关键行动之一.
61NSF和亚马逊正在合作62共同支持以人工智能公平为重点的研究,目标是为可信赖的人工智能系统做出贡献,这些系统易于接受和部署,以应对社会面临的重大挑战.
感兴趣的具体主题包括但不限于透明度,可解释性,问责制,潜在的不利偏见和影响,缓解策略,公平性验证和包容性考虑.
可解释性,公平性和透明度:最近的代理机构研发计划国家人工智能研发战略计划:2019–24–当AI代理人自主行动时,我们希望他们的行为符合我们对人类同胞的正式和非正式规范.
因此,作为基本的社会秩序力量,法律和道德既可以通知和判断人工智能系统的行为.
主要的研究需求包括理解AI的伦理,法律和社会影响,以及开发符合道德,法律和社会原则的AI设计方法.
还必须考虑隐私问题;关于这个问题的进一步信息可以在国家隐私研究战略中找到.
66与任何技术一样,人工智能的可接受用途将以法律和道德的原则为依据;挑战在于如何将这些原则应用于这项新技术,特别是那些涉及自治,代理和控制的技术.
如"强大和有益的人工智能的研究重点"所阐明的那样.
67为了构建运行良好的系统,我们当然需要确定每个应用程序域中的良好行为含义.
这种道德维度与可用的工程技术,这些技术的可靠性以及进行权衡的问题密切相关-计算机科学,机器学习和更广泛的AI专业知识的所有领域都是有价值的.
这一领域的研究可以受益于多学科的观点,涉及计算机科学,社会和行为科学,伦理学,生物医学,心理学,经济学,法律和政策研究专家.
需要在NITRD相关IT领域内外的领域(即信息技术以及前面提到的学科)进行进一步调查,以便为人工智能系统的研发和使用及其对社会的影响提供信息.
以下小节探讨了该领域的关键信息技术研究挑战.
66https://www.
nitrd.
gov/pubs/NationalPrivacyResearchStrategy.
pdf67"公开信:强有力和有益的人工智能的研究重点"(生命未来研究所):http://futureoflife.
org/ai-open-letter/.
2019年维持美国人工智能领导力行政命令强调,保持美国在人工智能方面的领导力需要共同努力,以促进技术和创新的进步,同时保护公民自由,隐私和美国价值观:1美国必须培养公众对人工智能技术的信任和信心,并在其应用中保护公民自由,隐私和美国价值观,以便充分发挥人工智能技术对美国人民的潜力.
需要更多的研发来开发人工智能架构,通过透明度和可解释性等技术机制将道德,法律和社会问题纳入其中.
这项研发将需要技术专家以及其他领域的利益相关者和专家之间的密切合作,包括社会和行为科学,法律,道德和哲学.
由于道德决策也可能严重依赖于上下文或应用程序,因此也可能需要与领域专家进行协作.
这种跨学科方法可以纳入AI的培训,设计,测试,评估和实施,以便理解和解释AI引起的决策和行动,并减轻意外后果.
因此,联邦机构应通过赞助研究和召集专家和利益相关者,继续促进不断增长的社区对这些问题的进一步研发的兴趣.
国家人工智能研发战略计划:2019–25–通过设计提高公平性,透明度和问责制人们对数据密集型人工智能算法对错误和误用的敏感性以及性别,年龄,种族或经济阶层可能产生的后果表示了许多担忧.
在这方面,适当收集和使用人工智能系统的数据是一项重要挑战.
然而,除了纯粹与数据相关的问题之外,人工智能的设计出现了更大的问题,这些问题本身就是公正,公平,透明和负责任的.
研究人员必须学习如何设计这些系统,以便他们的行动和决策是透明的,并且可以被人类轻易解释,因此可以检查他们可能包含的任何偏见,而不仅仅是学习和重复这些偏见.
关于如何表达和"编码"价值和信仰系统存在严重的知识问题.
科学家还必须研究在系统中设计正义和公平因素的程度,以及如何在当前工程技术的范围内实现这一点.
建立道德AI除了正义和公平的基本假设之外,人工智能系统是否能够表现出遵守一般道德原则的行为.
人工智能的进步如何在道德规范中提出新的"机器相关"问题,或者人工智能的使用可能被认为是不道德的道德本质上是一个哲学问题,而人工智能技术依赖于工程学,并受其限制.
因此,在技术可行的范围内,研究人员必须努力开发可验证地符合或符合现有法律,社会规范和道德规范的算法和架构-显然是一项非常具有挑战性的任务.
道德原则通常具有不同程度的模糊性,难以转化为精确的系统和算法设计.
当AI系统,尤其是新型自主决策算法,面临基于独立且可能相互冲突的价值体系的道德困境时,也会出现复杂情况.
道德问题因文化,宗教和信仰而异.
但是,可以制定可接受的道德参考框架来指导人工智能系统的推理和决策,以便解释和证明其结论和行动的合理性.
需要采用多学科方法来生成反映适当价值体系的培训数据集,包括表明在遇到困难的道德问题或具有相互冲突的价值时的首选行为的例子.
这些示例可以包括合法或道德的"极端案例",由对用户透明的结果或判断标记.
68人工智能需要适当的方法来解决基于价值的冲突,其中系统包含的原则可以解决严格规则不切实际的复杂情况的现实.
为道德AI设计架构必须在基础研究方面取得进一步进展,以确定如何最好地设计包含道德推理的AI系统架构.
已经提出了各种方法,例如将操作AI与监视器代理分开的双层监视器体系结构,监视器代理负责对任何操作操作进行道德或法律评估.
.
68另一种观点是安全工程是首选,其中使用AI代理体系结构的精确概念框架来确保AI行为是安全的并且对人类无害.
69第三种方法是使用集合理论原则与逻辑约束相结合来制定道德体系结构68A.
Etzioni和O.
Etzioni,"设计符合我们法律和价值观的人工智能系统",ACM通讯59(9)(2016):29-31.
69RYYampolsky,"人工智能安全工程:为什么机器伦理是一种错误的方法.
"在哲学和人工智能理论,编辑.
VCMuller(Heidelberg:SpringerVerlag,2013),389-396.
国家人工智能研发战略计划:2019–22–关于限制行动以符合道德原则的AI系统行为.
70随着人工智能系统变得更加通用,他们的架构可能会包含可以在多个判断级别上处理道德问题的子系统,包括:71快速反应模式匹配规则,用于描述和证明行动的较慢响应的协商推理,用于指示用户可信度的社交信号,以及在更长时间尺度上操作以使系统遵守文化规范的社交过程.
研究人员需要关注如何最好地解决符合道德,法律和社会目标的AI系统的整体设计.
70RCArkin,"管理法律行为:在混合协商/反应机器人架构中嵌入道德",佐治亚理工学院技术报告,GIT-GVU-07-11,2007.
71B.
Kuipers,"人类道德与机器人伦理",AAAI-16人工智能,伦理与社会研讨会,2016年;https://web.
eecs.
umich.
edu/~kuipers/papers/Kuipers-aaaiws-16.
pdf国家人工智能研发战略计划:2019–27–战略4:确保AI系统的安全性2019更新创建健壮且值得信赖的AI系统自2016年国家AI研发战略计划发布以来,人工智能安全和安全的科学和社会理解迅速增长.
这些新知识中的大部分都有助于发现新问题:现在更明显的是,人工智能系统如何能够做错事,学习错误信息或揭露错误信息,例如,通过对抗性示例,数据中毒和模型反演分别.
不幸的是,这些人工智能安全和安全问题的技术解决方案仍然难以捉摸.
为解决所有这些问题,必须在AI系统生命周期的所有阶段(从初始设计和数据/模型构建,到验证和验证,部署,操作和监控)考虑AI系统的安全性和安全性.
实际上,"设计安全(或安全)"的概念可能会产生一种错误的观念,即这些只是系统设计者的关注点;相反,它们必须在整个系统生命周期中考虑,而不仅仅是在设计阶段,因此必须是AI研发组合的重要组成部分.
当AI组件连接到其他系统或必须安全或安全的信息时,AI漏洞和性能要求(例如,当在大量数据上操作时,误报率和假阴性率非常低)72https://www.
transportation.
gov/av/373https://www.
nhtsa.
gov/automated-driving-systems/voluntary-safety-self-assessment74https://www.
iarpa.
gov/index.
php/research-programs/sails75https://www.
iarpa.
gov/index.
php/research-programs/trojai76https://www.
darpa.
mil/news-events/2019-02-0677"2018年国防部人工智能战略总结":https://media.
defense.
gov/2019/Feb/12/2002088963/-1/-1/1/SUMMARY-OF-DOD-AI-STRATEGY.
PDF.
自2016年国家AI研发战略计划发布以来,一些机构已开始支持战略4:DOT于2018年10月发布了新的联邦自动驾驶汽车指南,支持将自动化安全地集成到广泛的多式联运系统中.
为交通的未来做好准备:自动驾驶汽车3.
072推进DOT的自动驾驶汽车安全整合原则.
该文件还重申了先前的安全指南,提供了新的多模式安全指导,并概述了随着这项新技术的发展而与DOT合作的过程.
截至2019年5月,已有14家公司发布了自愿安全自我评估,详细说明了如何将安全性纳入其自动驾驶系统的设计和测试中.
73在2018年12月,IARPA宣布了两项关于人工智能安全的计划:安全,可靠,智能学习系统(SAILS)74和人工智能中的特洛伊木马(TrojAI).
75DARPA于2019年2月宣布另一项计划,即保证人工智能对欺骗行为的可靠性(GARD).
76这些计划旨在共同打击对AI系统的一系列攻击.
如战略3所述,国防部致力于"引领军事道德和人工智能安全",作为指导其加速采用人工智能系统的战略方法中列出的五项关键行动之一.
77人工智能安全与保障:最近的代理机构研发计划国家人工智能研发战略计划:2019–28–由较大的系统继承.
这些挑战并非一成不变;随着人工智能系统能力的不断提高,他们的复杂性可能会越来越高,这使得更难以正确的性能或信息的隐私得到验证和验证.
这种复杂性也可能使得越来越难以以证明人类用户高度信任的方式解释决策(见战略3).
让AI值得信赖-现在和未来-是一个关键问题,需要联邦政府的研发投资(见边栏),以及政府,行业,学术界和民间社会之间的合作.
工程值得信赖的AI系统可以从其他领域的安全工程中借用现有实践中受益,这些领域已经学会了如何解释非AI自主或半自治系统的潜在不当行为.
然而,特定于AI的问题意味着用于程序分析,测试,形式验证和综合的新技术对于确定基于AI的系统满足其规范(即系统完全按照预期执行的操作)至关重要.
而且没有了.
这些问题在基于人工智能的系统中会加剧,这些系统很容易被愚弄,逃避和误导,其方式可能具有深远的安全隐患.
一个新兴的研究领域是对抗性ML,它探讨了ML算法中的漏洞分析以及产生更强大学习的算法技术.
对ML的众所周知的攻击包括对抗性分类器规避攻击,其中攻击者改变行为以逃避被检测,以及中毒攻击,其中训练数据本身被破坏.
越来越需要研究系统地探索攻击ML和其他基于AI的系统的攻击者的空间,并设计能够提供针对攻击类别的可证实的鲁棒性保证的算法.
必须开发方法以确保AI的创建,评估,部署和控制的安全性,并且这些方法必须扩展以匹配AI的能力和复杂性.
评估这些方法将需要新的指标,控制框架和基准,以测试和评估日益强大的系统的安全性.
方法和指标都必须包含人为因素,人类设计者目标定义的安全AI目标,人类用户习惯定义的安全AI操作,以及人类评估者理解的安全AI指标.
为人工智能系统的安全性制定人为驱动和人类可理解的方法和指标将使政策制定者,私营部门和公众能够准确地判断不断变化的人工智能安全格局并在其中适当地进行.
在人工智能系统得到广泛使用之前,需要确保系统以受控方式安全可靠地运行.
需要进行研究以解决创建可靠,可靠和值得信赖的AI系统的挑战.
与其他复杂系统一样,AI系统面临重要的安全和安全挑战,原因如下:78复杂和不确定的环境:在许多情况下,AI系统被设计为在复杂环境中运行,具有大量潜在状态,无法对其进行详尽检查或测试.
系统可能面临在设计过程中从未考虑过的条件.
紧急行为:对于在部署后学习的AI系统,系统的行为可能主要由在无监督条件下学习的时段决定.
在这种情况下,可能难以预测系统的行为.
目标错误指定:由于难以将人类目标转换为计算机指令,因此为AI系统编程的目标可能与程序员所预期的目标不匹配.
78J.
Bornstein,"DoDAutonomyRoadmap-AutonomyCommunityofInterest",NDIA第16届年度科学与工程技术会议上的演讲,2015年3月.
国家人工智能研发战略计划:2019–29–人机交互:在许多情况下,人工智能系统的性能受人类交互的影响很大.
在这些情况下,人类反应的变化可能会影响系统的安全性.
79为了解决这些问题和其他问题,需要额外的投资来推进人工智能的安全和保障,80包括可解释性和透明度,信任,验证和验证,针对攻击的安全性以及长期AI安全性和价值调整.
提高可解释性和透明度一项关键的研究挑战是提高AI的"可解释性"或"透明度".
许多算法(包括基于深度学习的算法)对用户来说是不透明的,几乎没有用于解释其结果的现有机制.
对于诸如医疗保健之类的领域而言,这尤其成问题,其中医生需要解释来证明特定诊断或治疗过程的合理性.
诸如决策树感应之类的AI技术提供内置解释,但通常不太准确.
因此,研究人员必须开发透明的系统,并且本质上能够向用户解释其结果的原因.
建立信任为了获得信任,AI系统设计人员需要创建具有信息性,用户友好界面的准确,可靠的系统,而操作员必须花时间进行充分的培训,以了解系统操作和性能限制.
用户广泛信任的复杂系统,例如车辆的手动控制,往往是透明的(系统以用户可见的方式运行),可信(系统的输出被用户接受),可审计(系统可以被评估),可靠(系统充当用户的意图)和可恢复(用户可以在需要时恢复控制).
当前和未来AI系统面临的重大挑战仍然是软件生产技术的质量不稳定.
随着进步带来人与人工智能系统之间更大的联系,信任领域的挑战是跟上变化和增加能力的步伐,预测采用和长期使用的技术进步,并为最佳研究制定治理原则和政策.
设计,施工和使用的实践,包括安全操作的适当操作员培训.
加强验证和验证AI系统的验证和验证需要新方法.
"验证"确定系统符合正式规范,而"验证"确定系统满足用户的操作需求.
安全的AI系统可能需要新的评估方法(确定系统是否出现故障,可能是在超出预期参数的情况下运行),诊断(确定故障原因)和维修(调整系统以解决故障).
对于在长时间内自主运行的系统,系统设计人员可能没有考虑系统将遇到的每种情况.
此类系统可能需要具备自我评估,自我诊断和自我修复的能力,以便稳健可靠.
79JMBradshaw,RRHoffman,M.
Johnson和DDWoods,"自治系统的七大致命神话",IEEEIntelligentSystems28(3)(2013):54-61.
80例如:见D.
Amodei,C.
Olah,J.
Steinhardt,P.
Christiano,J.
Schulman和D.
Mane,"人工智能安全中的具体问题",2016年,ARXIV:160606565V2;S.
Russell,D.
Dewey和M.
Tegmark,"稳健和有益的人工智能的研究重点",2016年,arXiv:1602.
03506;TGDietterich和EJHorvitz,"对AI的关注的兴起:反思与方向",ACM通讯,58(10)(2015);和K.
Sotala和R.
Yampolskiy,"应对灾难性AGI风险:一项调查",PhysicaScripta,90(1),2014年12月19日.
国家人工智能研发战略计划:2019–30–防范攻击嵌入关键系统的AI必须是强大的,以便处理事故,但也应该对各种有意的网络攻击安全.
安全工程涉及了解系统的漏洞以及可能有兴趣攻击它的行为者的行为.
在NITRD2016联邦网络安全研发战略计划中更详细地讨论了网络安全研发需求,81一些网络安全风险是AI系统特有的.
例如,一个关键的研究领域是"对抗性机器学习",它探讨了通过"污染"训练数据,修改算法或对对象进行细微更改以防止其被正确识别而影响AI系统的程度.
(例如,欺骗面部识别系统的假肢).
在需要高度自治的网络安全系统中实施AI也是一个需要进一步研究的领域.
最近在这一领域工作的一个例子是DARPA的网络大挑战,其中涉及AI代理人自主分析和打击网络攻击.
82实现长期人工智能安全和价值调整AI系统最终可能具有"递归自我改进"的能力,其中软件本身而不是人类程序员进行了大量的软件修改.
为了确保自我修改系统的安全性,需要进一步开展研究:自我监控架构,检查系统与人类设计师的原始目标的行为一致性;禁止在评估系统时释放系统的限制策略;价值学习,其中用户的价值观,目标或意图可以由系统推断;和价值框架,可以证明自我修改.
81https://www.
nitrd.
gov/pubs/2016-Federal-Cybersecurity-Research-and-Development-Strategic-Plan.
pdf;这是在2019年更新.
82https://archive.
darpa.
mil/CyberGrandChallenge_CompetitorSite/国家人工智能研发战略计划:2019–31–策略5:为人工智能培训和测试开发共享的公共数据集和环境2019更新增加对数据集和相关挑战的访问在2016年国家AI研发战略计划发布时,公开可用的数据集和环境已经在推动AI研发方面发挥了关键作用,特别是在计算机视觉,自然语言处理和语音识别等领域.
ImageNet84超过1400万个标记对象,以及相关的计算机视觉社区挑战(例如,ImageNet大规模视觉识别挑战85评估对象检测和图像分类的算法),在社区中发挥了特别重要的作用.
随着ML的转化应用程序在医疗保健,医学,智能和互联社区等众多应用领域中被发现,对特定领域领域的公开数据集的需求也在增长.
数据集和模型的重要性-特别是联邦政府的数据集和模型-在2019年维持美国人工智能领导力行政命令中明确提出:1所有机构的负责人都应审查其联邦数据和模型,以确定更大的非联邦AI研究社区以有利于该社区的方式增加访问和使用的机会,同时保护安全,安全,隐私和机密性.
具体而言,代理商应改进数据和模型库存文档以实现发现和可用性,并且应该83https://insight.
shrp2nds.
us/84http://www.
image-net.
org/85http://www.
image-net.
org/challenges/LSVRC/自2016年国家人工智能研发战略计划发布以来,一些机构已开始支持战略5:DOT赞助了第二次战略公路研究计划(SHRP2)自然驾驶研究(NDS),83记录超过超过3,400名司机和车辆乘坐540万人次.
车载数据采集系统(DAS)单元收集并存储来自前方雷达,四个摄像机,加速度计,车辆网络信息,地理定位系统和车载车道跟踪器的数据.
在参与者的车辆运行时,DAS的数据不断记录.
虽然NDS数据的摘要是公开的,但访问详细数据集需要合格的研究伦理培训.
VADataCommons正在创建世界上最大的链接医学基因组数据集,其中包含用于启用ML和AI的工具,并由退伍军人的偏好指导.
这项工作正在利用适用的NIST标准,法律和行政命令.
GSA(一般服务管理局)正致力于将云计算资源用于联邦政府资助的AI研发.
位于GSA的Data.
gov和code.
gov包含来自各机构的超过246,000个数据集和代码,并自动收集各机构发布的数据集.
NIH发现,实验和可持续发展的科学和技术研究基础设施(STRIDES)倡议与行业领先的云服务提供商建立了合作关系,使研究人员能够访问由NIH资助并存储在云环境中的主要数据资产.
用于AI培训和测试的共享公共数据集和环境:最近的代理研发计划国家人工智能研发战略计划:2019–32–根据AI研究社区的反馈,优先考虑改进AI数据和模型的访问和质量.
新的NSTC开放科学小组委员会于2018年成立,旨在协调联邦在开放和FAIR(可查找,可访问,可互操作和可重用)数据方面的努力.
将需要研发投资来开发工具和资源,以便更容易识别,使用和操纵相关数据集(包括联邦数据集),验证数据来源,并尊重适当的使用政策.
许多这些数据集本身在人工智能环境中的使用可能有限,而无需在标签和管理方面进行投资.
联邦机构应该与AI利益相关方合作并确保适当审查的数据集和模型已经准备好并且适合使用,并且随着标准和规范的发展而得到维护.
最终,在记录数据集和模型出处时开发和采用最佳实践和标准将提高可靠性和负责任地使用AI技术.
自2016年以来,人们对数据内容的担忧也越来越多,例如潜在的偏见(见战略3)86,87或私人信息泄露.
2016年国家AI研发战略计划指出,"数据集的开发和共享必须遵循适用的法律法规,并以道德的方式进行.
"DOT支持的InSight项目提供了对自然驾驶研究期间收集的数据的精心组织的访问.
(见附文).
2016年国家AI研发战略计划还指出,需要新的"技术来确保数据的安全共享,因为数据所有者在与研究团体共享数据时承担风险.
"例如,CryptoNets88允许神经网络对加密数据进行操作,确保数据保密,因为神经网络中不需要解密密钥.
研究人员还开始开发新的ML技术,使用差异隐私框架为使用的数据提供可量化的隐私保证.
89同时,隐私方法必须保持足够的可解释性和透明度,以帮助研究人员纠正它们并使其安全,有效和准确.
此外,AI可以揭示超出原始或预期范围的发现;因此,研究人员必须认识到对手获取数据或发现的潜在危险.
如果没有将计算资源用于大规模公共数据集的能力,那么单独使用数据几乎没有用处.
计算资源对人工智能研发的重要性在2019年"维持美国人工智能领导力行政命令"中得到了体现:1国防,商业,卫生和人类服务部门和能源部长,美国国家航空航天局局长和国家科学基金会主任应在适当和适用的范围内,优先考虑高分配-通过以下方式为AI相关应用程序提供的性能计算资源:(i)增加对资源和资源储备的自由分配的分配;或(ii)任何其他适当的机制.
86EmilyM.
Bender和BatyaFriedman,"NLP的数据陈述:趋向于减轻系统偏差和实现更好的科学",计算语言学协会的交易6(2018):587-604.
87AylinCaliskan,JoannaJ.
Bryson和ArvindNarayanan,"自动从语料库中衍生的语义包含类似人类的偏见",Science356(6334):183-186,2017年4月14日.
88RanGilad-Bachrach,NathanDowlin,KimLaine,KristinLauter,MichaelNaehrig,JohnWernsing,"CryptoNets:将神经网络应用于高吞吐量和高精度的加密数据",2016年国际机器学习会议48:201-210;http://proceedings.
mlr.
press/v48/.
89MartinAbadi,AndyChu,IanGoodfellow,H.
BrendanMcMahan,IlyaMironov,KunalTalwar和LiZhang,"深度学习与差异隐私",第23届ACM计算机与通信安全会议,"2016:308-318.
国家人工智能研发战略计划:2019–33–人工智能的好处将继续增加,但仅限于开发和提供人工智能的培训和测试资源.
训练数据集和其他资源的多样性,深度,质量和准确性会显着影响AI性能.
许多不同的AI技术需要高质量的数据用于培训和测试,以及动态,交互式测试平台和模拟环境.
不仅仅是一个技术问题,这是一个重要的"公益"挑战,因为如果人工智能培训和测试仅限于少数已经拥有宝贵数据集和资源的实体,那么进步将受到影响,但我们必须同时尊重商业和个人权利和对数据的兴趣.
需要开展研究,为各种AI应用开发高质量的数据集和环境,并负责访问良好的数据集,测试和培训资源.
还需要额外的开源软件库和工具包来加速AI研发的进程.
以下小节概述了这些重要的关键领域.
开发和提供各种数据集,以满足各种AI兴趣和应用的需求人工智能培训和测试数据集的完整性和可用性对于确保科学可靠的结果至关重要.
支持数字领域可重复研究所需的技术和社会技术基础设施已被认为是一项重要挑战-对人工智能技术也至关重要.
缺乏经过审查和公开可用的数据集以及确定的可重复性的数据集是人工智能自信进步的关键因素.
90与其他数据密集型科学一样,捕获数据来源至关重要.
研究人员必须能够使用相同和不同的数据集重现结果.
数据集必须代表具有挑战性的真实世界应用程序,而不仅仅是简化版本.
为了快速取得进展,应该把重点放在提供政府持有的现有数据集,可以用联邦资金开发的数据集,以及尽可能用工业持有的数据集.
AI挑战的机器学习方面通常与"大数据"分析相关联.
考虑到各种相关数据集,对非结构化或半结构化数据进行适当的表示,访问和分析仍然是一个日益严峻的挑战.
如何以绝对和相对(依赖于上下文)的术语表示数据当前的真实数据库可能非常容易受到不一致,不完整和嘈杂的数据的影响.
因此,许多数据预处理技术(例如,数据清理,集成,转换,缩减和表示)对于为AI应用程序建立有用的数据集非常重要.
数据预处理如何影响数据质量,尤其是在执行其他分析时90为此,2016年,情报高级研究项目活动发布了关于新型训练数据集和环境的信息请求,以推进AI.
看到https://iarpa.
gov/index.
php/working-with-iarpa/请求换信息/新颖,训练数据集和的环境中对提前人工智能.
和:.
.
.
.
.
.
专责委员会应与总务管理局(GSA)协调,向总统提交报告,就更好地使用云计算资源进行联邦政府资助的人工智能研发提出建议.
对许多AI挑战的计算能力的需求一直在迅速增加.
32联邦资助可为联邦政府资助的研究提供计算能力.
然而,一些公司和大学可能有额外的计算需求.
总体而言,国家需要研究和投资共享计算资源以促进人工智能研发.
国家人工智能研发战略计划:2019–34–鼓励共享AI数据集-尤其是政府资助的研究-可能会激发创新的AI方法和解决方案.
但是,需要技术来确保数据的安全共享,因为数据所有者在与研究社区共享数据时会承担风险.
数据集开发和共享还必须遵守适用的法律和法规,并以道德的方式进行.
风险可能以各种方式出现:不恰当地使用数据集,不准确或不恰当的披露,以及数据去识别技术的限制,以确保隐私和机密性保护.
根据商业和公共利益制定培训和测试资源随着全球数据,数据源和信息技术的不断爆炸,数据集的数量和规模都在不断增加.
分析数据的技术和技术无法跟上大量原始信息源.
数据采集,策展,分析和可视化都是关键的研究挑战,从大量数据中提取有价值的知识所需的科学是滞后的.
虽然存在数据存储库,但它们通常无法处理数据集的扩展,数据来源信息有限,并且不支持语义丰富的数据搜索.
需要动态,灵活的存储库.
支持人工智能研究需求所需的开放/共享基础设施计划的一个例子是国土安全部(DHS)制定的IMPACT计划(网络风险与信任政策与分析信息市场).
91该计划通过协调和开发实际数据和信息共享功能(包括工具,模型和方法)来支持全球网络安全风险研究工作.
IMPACT还支持国际网络安全研发社区,关键基础设施提供商及其政府支持者之间的经验数据共享.
AI研发将受益于所有AI应用程序中的可比程序.
开发开源软件库和工具包开源软件库和工具包的可用性越来越高,可以为任何具有Internet连接的开发人员提供基于图像的AI技术.
Weka工具包等资源,92槌,93和OpenNLP,94在许多其他人中,加速了人工智能的开发和应用.
开发工具(包括免费或低成本的代码存储库和版本控制系统)以及免费或低成本的开发语言(例如,R,Octave和Python)为使用和扩展这些库提供了较低的障碍.
此外,对于那些可能不想直接集成这些库的人,可以使用任意数量的基于云的机器学习服务,这些服务可以通过需要很少或不需要编程的低延迟Web协议按需执行图像分类.
最后,许多这些Web服务还提供专用硬件的使用,包括基于GPU的系统.
可以合理地假设用于AI算法的专用硬件(包括神经形态处理器)也将通过这些服务广泛使用.
这些资源共同提供了一个AI技术基础设施,鼓励市场创新,允许企业家开发解决方案,解决窄域问题,无需昂贵的硬件或软件,无需高水平的AI专业知识,并允许按需快速扩大系统规模.
对于狭窄的AI领域,与许多其他技术领域相比,市场创新的障碍极低.
91https://www.
dhs.
gov/csd-impact92https://sourceforge.
net/projects/weka/93http://mallet.
cs.
umass.
edu94https://opennlp.
apache.
org国家人工智能研发战略计划:2019–35–为了帮助支持该领域持续的高水平创新,美国政府可以加大开放,支持和使用开放式人工智能技术的力度.
特别有益的是使用标准化或开放格式的开放资源和用于表示语义信息的开放标准,包括可用时的领域本体.
政府还可以通过加速在政府内部使用开放式人工智能技术来鼓励更多地采用开放的人工智能资源,从而有助于保持创新者进入门槛较低.
政府应尽可能为开源项目提供算法和软件.
由于政府有特定的关注点,例如更加重视数据隐私和安全性,政府可能有必要制定机制来减轻政府采用人工智能系统的速度.
例如,创建一个可以跨政府机构执行"水平扫描"以找到部门内特定AI应用领域的工作组,然后确定需要解决的特定问题以允许采用此类技术可能是有用的.
这些机构.
国家人工智能研发战略计划:2019–36–策略6:通过标准和基准测量和评估AI技术2019更新支持开发人工智能技术标准和相关工具2016年国家人工智能研发战略计划指出,"人工智能社区的标准,基准,测试平台及其采用对于指导和促进人工智能技术的研发至关重要.
"在此期间的三年中,对标准和基准的重视程度不断提高在美国和全球.
2019年"维持美国人工智能领导力"行政命令明确提出了这些标准的重要性:1.
.
.
.
.
.
商务部长通过[NIST]主任,应制定联邦参与技术标准和相关工具开发的计划,以支持使用AI技术的可靠,稳健和可靠的系统.
随着人工智能创新可能影响社会的所有部门和领域,许多标准制定组织正在进行新的与人工智能相关的考虑和工作项目,包括与人工智能道德和可信赖的人工智能系统相关的活动(见战略3).
国际标准化组织(ISO)和国际电工委员会(IEC)召集了一个关于AI的联合技术小组委员会(ISO/IEC联合技术委员会1,小组委员会)42关于人工智能95)制定人工智能系统标准和相关考虑因素.
至关重要的是,联邦,工业和学术研究人员应继续为这些活动提供信息,特别是当人工智能的进步和系统进入与政府机构的使命相一致的交通,医疗保健和食品等领域时.
自2016年以来,人工智能相关标准活动的激增超过了新的以人工智能为基础的基准和评估的推出,特别是与人工智能系统的可信度相关.
在里面95https://www.
iso.
org/committee/6794475.
html96https://home.
treasury.
gov/policy-issues/international/g-7-and-g-20自2016年国家AI研发战略计划发布以来,NIST特别为战略6开展了工作:NIST参与ISO/IECJTC1SC42关于人工智能的标准化计划.
95NIST专家是SC42中大数据工作的召集人.
美国驻SC42代表团包括NIST和其他联邦机构专家,以及来自工业界和学术界的代表.
美国对SC42的投入由国际信息技术标准委员会(INCITS)提供便利.
NIST员工通过标准组织参与其他AI标准活动,例如美国机械工程师协会,IEEE和ISO/IEC.
他们的活动涉及诸如先进制造的计算建模,机器人和自动化的本体,个人数据隐私和算法偏差等主题.
NIST专家正在提高人们对多边论坛中人工智能共识标准重要性的认识,包括G20和G7等机构.
96NIST带来了独特的联邦政府专业知识,可以在实践中进行政策讨论,特别是通过与私营部门的密切合作.
同样,NIST将其标准和相关经验用于政府间双边讨论.
标准,基准和相关工具:最近的机构研发计划国家人工智能研发战略计划:2019–37–AI社区的标准,基准,测试平台及其采用对于指导和促进AI技术的研发至关重要.
以下小节概述了必须取得进一步进展的领域.
制定广泛的AI标准必须加快标准的开发,以跟上快速发展的能力和扩展AI应用领域的步伐.
标准提供了可以一致使用的要求,规范,指南或特性,以确保AI技术满足功能和互操作性的关键目标,并且可靠且安全地执行.
采用标准为技术进步带来可信度,并促进扩展的可互操作市场.
已经开发的AI相关标准的一个示例是由电气和电子工程师协会开发的P1872-2015(机器人和自动化的标准本体).
该标准提供了表示知识的系统方法和一组通用的术语和定义.
这些允许在人类,机器人和其他人工系统之间进行明确的知识转移,并为人工智能技术应用AI技术提供基础.
人工智能的所有子域都需要在AI标准开发方面进行额外的工作.
需要标准来解决:软件工程:管理系统复杂性,维护,安全性以及监控和控制紧急行为;性能:确保准确性,可靠性,稳健性,可访问性和可扩展性;度量标准:量化影响性能和符合标准的因素;安全性:评估系统的风险管理和危害分析,人机交互,控制系统和法规遵从性;可用性:确保界面和控件的有效性,高效性和直观性;互操作性:通过标准和兼容接口定义可互换的组件,数据和事务模型;安全性:解决信息的机密性,完整性和可用性以及网络安全问题;隐私:控制在处理,运输或存储时保护信息;97https://mlperf.
org/98https://dawn.
cs.
stanford.
edu/benchmark/然而,干预时间,基准数据集中的公平性和偏见的考虑变得越来越重要,研究人员正在寻求新的面部识别数据集,以寻求最小化偏差.
测试AI算法的应用程序级性能(例如,分类算法的假阳性或假阴性率)和量化AI软件和硬件系统的计算级性能的基准测试的基准测试更为丰富.
最近的两项活动是MLPerf97和DAWNbench.
98评估,促进和确保AI可信赖性的所有方面需要通过基准和标准来衡量和评估AI技术性能.
除了安全,可靠,可靠,有弹性,可解释和透明之外,值得信赖的AI必须保护隐私,同时检测并避免不适当的偏见.
随着人工智能技术的发展,开发新指标和测试要求以验证这些基本特征的需求也将随之增加.
国家人工智能研发战略计划:2019–38–可追溯性:提供事件记录(实施,测试和完成)以及数据管理;和域:定义特定于域的标准词典和相应的框架.
建立AI技术基准由测试和评估组成的基准为制定标准和评估标准的合规性提供了量化指标.
基准通过促进旨在解决战略性选择方案的进步来推动创新;他们还提供客观数据来跟踪人工智能科学和技术的发展.
为了有效地评估AI技术,必须开发和标准化相关且有效的测试方法和指标.
标准测试方法将规定评估,比较和管理AI技术性能的协议和程序.
需要使用标准指标来定义可量化的度量,以便表征AI技术,包括但不限于:准确性,复杂性,信任和能力,风险和不确定性,可解释性,意外偏差,与人类绩效的比较以及经济影响.
值得注意的是,基准测试是数据驱动的.
策略5讨论了数据集对培训和测试的重要性.
作为与AI相关的基准测试的成功范例,美国国家标准与技术研究院开发了一套全面的标准测试方法和相关的性能指标,以评估应急响应机器人的关键功能.
目的是通过利用使用标准测试方法捕获的机器人能力的统计上重要的数据来促进不同机器人模型的定量比较.
这些比较可以指导购买决策并帮助开发人员了解部署功能.
通过ASTM国际安全应用标准委员会对机器人操作设备(称为标准E54.
08.
01)进行了标准化测试.
99测试方法的版本用于通过RoboCup救援机器人联盟比赛挑战研究社区,100强调自主能力.
另一个例子是IEEEAgileRoboticsforIndustrialAutomationCompetition(ARIAC),101IEEE与NIST的共同努力,102通过利用人工智能和机器人规划的最新进展,提高机器人的灵活性.
本次竞赛的核心焦点是测试工业机器人系统的灵活性,目标是使车间的人员更高效,更自主,并且需要更少的车间工人时间.
虽然这些努力为推动人工智能基准测试提供了坚实的基础,但它们受到特定领域的限制.
更广泛的领域需要额外的标准,测试平台和基准,以确保AI解决方案广泛适用并广泛采用.
增加AI测试平台的可用性"未来的网络实验"报告中指出了测试平台的重要性:"测试平台是必不可少的,以便研究人员可以使用实际的操作数据来模拟和运行真实系统的实验.
.
.
.
.
.
以及良好测试环境中的情景.
"103有足够的测试台是一个992019更新:由此产生的测试方法现在是ASTM国际标准委员会在国土安全响应机器人应用程序中发布的标准(称为E54.
09).
100http://www.
robocup2016.
org/en/101http://robotagility.
wixsite.
com/competition1022019年更新:IEEE不再是ARIAC的合作伙伴,现已进入第三年.
103SRI国际和南加州大学信息科学研究所,"未来网络安全实验(CEF):催化新一代实验性网络安全研究",最终报告,2015年7月31日.
国家人工智能研发战略计划:2019–39–需要跨越AI的所有领域.
政府拥有大量政府独有的任务敏感数据,但大部分数据无法分发给外部研究界.
可以为学术和工业研究人员建立适当的计划,以便在特定机构建立的安全和策划的测试平台环境中进行研究.
AI模型和实验方法可以由研究团体通过访问这些测试环境来共享和验证,为AI科学家,工程师和学生提供独特的研究机会.
让AI社区参与标准和基准测试需要政府的领导和协调来推动标准化并鼓励其在政府,学术界和工业界广泛使用.
由用户,行业,学术界和政府组成的AI社区必须充满活力,参与制定标准和基准计划.
由于每个政府机构根据其角色和使命以不同方式参与社区,因此可以通过协调来利用社区互动,以加强其影响.
需要这种协调来集体收集用户驱动的需求,预测开发人员驱动的标准,并促进教育机会.
用户驱动的需求决定了挑战问题的目标和设计,并使技术评估成为可能.
拥有社区基准专注于研发,以确定进度,缩小差距,并针对特定问题推动创新解决方案.
这些基准必须包括定义和分配基本事实的方法.
基准模拟和分析工具的创建也将加速AI的发展.
这些基准测试的结果还有助于将正确的技术与用户的需求相匹配,形成符合标准的客观标准,合格的产品列表和潜在的源选择.
工业界和学术界是新兴AI技术的主要来源.
促进和协调他们参与标准和基准活动至关重要.
随着解决方案的出现,通过分享技术架构的共同愿景,开发新兴标准的参考实现以显示可行性,以及进行预竞争性测试以确保高质量和可互操作的解决方案,以及为了预测开发人员和用户驱动的标准,有很多机会.
开发技术应用的最佳实践.
高度影响,基于社区,与AI相关的基准程序的一个成功范例是文本检索会议(TREC),104这是由NIST于1992年启动的,旨在提供大规模评估信息检索方法所需的基础设施.
超过250个团体参加了TREC,包括大小学术和商业组织.
TREC提出的标准的,广泛可用的,精心构建的数据集已被用于振兴信息检索研究.
105,106第二个例子是应用于生物识别技术的机器视觉领域的NIST定期基准程序,107特别是面部识别.
108这开始于1993年的人脸识别技术(FERET)评估,该评估提供了面部照片的标准数据集,旨在支持人脸识别算法开发以及评估协议.
多年来,这项努力已经发展成为Face104http://trec.
nist.
gov105EMVoorhees和DKHarman,TRECExstrationmentandEvaluationinInformationRetrieval(Cambridge:MITPress,2005).
106http://googleblog.
blogspot.
com/2008/03/why-data-matters.
html107http://biometrics.
nist.
gov108http://face.
nist.
gov国家人工智能研发战略计划:2019–40–认可供应商测试(FRVT),109涉及数据集的分发,托管挑战问题以及进行隔离技术评估.
该基准程序为面部识别技术的改进做出了巨大贡献.
TREC和FRVT都可以作为有效的AI相关社区基准活动的例子,但AI的其他领域也需要类似的努力.
值得注意的是,制定和采用标准以及参与基准活动需要付出代价.
研发组织在看到重大利益时会受到激励.
更新各机构的采购流程,以便在提案请求中包含AI标准的特定要求,这将鼓励社区进一步参与标准制定和采用.
基于社区的基准,如TREC和FRVT,还通过提供各种类型的培训和测试数据来降低障碍并加强激励,否则无法获取,促进技术开发人员之间的良性竞争,以推动最佳算法,并提供客观和比较性能指标用于相关的源选择.
109PJPhillips,"改善人脸识别技术",计算机44(3)(2011):84-96.
国家人工智能研发战略计划:2019–41–战略7:更好地了解国家AI研发人员的需求2019更新推动人工智能研发人员,包括那些致力于人工智能系统和与他们一起工作的人员,以维持美国的领导地位自2016年国家AI研发战略计划发布以来,对AI研究人员和从业人员的需求迅速增长.
研究表明,在未来十年内,招聘机会的数量有望增加到数百万.
作为一个数据点,美国劳工统计局预计,从2016年到2026年,计算机和信息科学家和工程师的职位数量将增长19%,几乎是所有职业平均数的三倍.
111此外,到2028年,人工智能研究人员预计仅贡献于G20国家智能技术承诺的11.
5万亿美元的累积增长.
112美国学术机构正在努力跟上学生兴趣和人工智能入学的爆炸性增长.
113,114,115与此同时,行业凭借其持续的财务支持以及对先进计算设施和数据集的访问,对学术研究和教学人才产生了强大的影响.
116在人工智能中保持强大的学术研究生态系统至关重要,与生产者研发部门合作,可以继续带来巨大的回报117通过促进国民健康,繁荣和福利,确保国防.
110https://www.
krellinst.
org/csgf/math-cs111https://www.
bls.
gov/ooh/computer-and-information-technology/computer-and-information-research-scientists.
htm112HTTPS://www.
ACCENTURU.
COM/T20180920T094705ZW/USEn/A/NNMID/思维领导资产/PDF/埃森哲教育和技术技能,研究.
pdf113https://cra.
org/data/generation-cs/114https://cra.
org/wp-content/uploads/2018/05/2017-Taulbee-Survey-Report.
pdf115http://web.
cs.
wpi.
edu/~cew/papers/CSareas19.
pdf116https://www.
nitrd.
gov/rfi/ai/2018/AI-RFI-Response-2018-Yolanda-Gil-AAAI.
pdf117https://www.
nap.
edu/catalog/13427/continuing-innovation-in-information-technology自2016年国家AI研发战略计划发布以来,一些机构已开始支持战略7:除了通过标准的人工智能研究资助支持本科生和研究生之外,各机构还在其研究生奖学金计划中优先考虑计算和数据支持的科学和工程.
例如,在2018年,DOE为其计算科学研究生奖学金计划增加了一条新的轨道.
该课程支持学生攻读应用数学,统计学或计算机科学的高级学位,并促进更有效地使用高性能系统,包括AI,ML和深度学习领域.
44,110同样在2018年,NSF开始在其研究生研究奖学金计划的获奖者的子集中优先考虑计算和数据支持的科学和工程.
人口普查局创建了统计数据现代化(SDM)项目,以使其劳动力,运营和技术达到当前的最新水平,并为当今数据驱动型社会中的统计机构设定标准.
SDM的劳动力转换组件将雇用具有新方法和分析专业知识的新数据科学家,包括使用AI方法和工具来处理和分析大数据.
劳动力转型还将解决当前数据科学人员的技能提升问题.
国家人工智能研发人员:近期的机构活动国家人工智能研发战略计划:2019–42–自2016年国家人工智能研发战略计划发布以来的三年中,各种报告呼吁继续支持各级计算机科学教材和教师专业发展的发展.
几十年来,K-12层面需要强调为国家人工智能研究人员提供支持.
118在本科阶段,考虑到跨学科的计算越来越重要,需要将重点放在将先进的计算技能和方法与来自其他学科的特定领域知识相结合.
119研究生阶段也需要持续的支持,学生们正在进行ML和AI的基础研究.
实际上,2019年维持美国人工智能领导力的行政命令要求这样做:1提供教育补助金的执行机构负责人应在符合适用法律的范围内,将人工智能视为现有联邦奖学金和服务计划中的优先领域.
.
.
.
.
.
[包括]A)高中,本科和研究生奖学金;替代教育;和培训计划;(B)承认和资助进行人工智能研发的早期职业大学教师的计划,包括通过总统奖励和表彰;(C)服务计划奖学金;(D)美国武装部队的直接调试方案;(E)支持制定教学计划和课程的计划,鼓励将人工智能技术纳入课程,以促进正规和非正规教育和培训的个性化和适应性学习体验.
更广泛地说,在联邦政府2018年12月的科学,技术,工程和数学(STEM)教育五年战略计划中,也突出了对计算思维的坚实基础的需求,包括通过计算机科学教育.
120此外,必须扩大传统上在计算机和相关领域中代表性不足的群体的参与.
最后,AI研发人员将由多学科团队组成,不仅包括计算机和信息科学家和工程师,还包括其他领域的专家,这些领域是人工智能和ML创新及其应用的关键,包括认知科学和心理学,经济学和博弈论,工程学和控制理论,伦理学,语言学,数学,哲学以及可能应用AI的许多领域.
联邦机构优先考虑各级培训和奖学金计划,通过学徒,技能计划,奖学金和相关学科的课程工作,为员工提供必要的人工智能研发技能(见附文).
这些培训机会针对的是为AI研发创新做出贡献的科学家和工程师以及可能拥有相关领域知识的AI研发用户.
就前者而言,如战略1所述,联邦对人工智能研发的长期投资进一步支持了这一劳动力的增长,既培养了下一代研究人员,又使教师职位对当前的研究生和博士后更具吸引力.
学生们.
就后者而言,新计划正在为人工智能系统的当前和未来用户带来与人工智能相关的技能(见附文).
因此,联邦机构必须继续战略性地培养跨越多个学科和技能类别的人工智能研发人员的专业知识,以确保持续的国家领导.
118https://github.
com/touretzkyds/ai4k12/wiki119https://www.
nap.
edu/catalog/24926/assessing-and-responding-to-the-growth-of-computer-science-本科招生120https://www.
whitehouse.
gov/wp-content/uploads/2018/12/STEM-Education-Strategic-Plan-2018.
pdf国家人工智能研发战略计划:2019–43–实现本战略中概述的所需AI研发进展将需要足够的AI研发人员.
人工智能研发领域最强大的国家将在未来的自动化领域建立领先地位.
他们将成为算法创建和开发等能力的领跑者;能力论证;和商业化.
发展技术专长将为这些进步提供基础.
虽然目前没有正式的人工智能劳动力数据,但商业和学术部门最近的大量报告表明,人工智能现有专家的数量不断增加.
据报道AI专家供不应求,121需求预计将继续升级.
122据报道,高科技公司正在投入大量资源来招聘具有AI专业知识的教师和学生.
123据报道,大学和工业正在招募和留住人才.
124需要进一步的研究,以更好地了解人工智能研发的当前和未来国家劳动力需求.
需要数据来描述AI研发人员的当前状态,包括学术界,政府和行业的需求.
研究应该探索AI工作场所的供需力量,以帮助预测未来的劳动力需求.
需要了解预计的AI研发劳动力管道.
应包括对教育途径和潜在再培训机会的考虑.
还应探索多样性问题,因为研究表明,多样化的信息技术劳动力可以带来改善的结果.
125一旦更好地理解当前和未来的AI研发人员需求,就可以考虑采取适当的计划和行动来解决任何现有或预期的劳动力挑战.
121"初创公司旨在利用深度学习技能差距",麻省理工学院技术评论,2016年1月6日.
122"人才攫取引发兴奋和担忧,"大自然,2016年4月26日.
123"人工智能专家需求量大","华尔街日报",2015年5月1日.
124"百万美元的婴儿:随着硅谷争夺人才,大学努力抓住他们的明星,"经济学人,2016年4月2日.
125JWMoody,CMBeise,ABWoszczynski和MEMyers,"多样性与信息技术劳动力:障碍与机遇",计算机信息系统期刊43(2003):63-71.
国家人工智能研发战略计划:2019–44–战略8:扩大公私合作伙伴关系,加速人工智能的发展战略8在2019年是新的,反映了实现人工智能研发的公私伙伴关系日益增长的重要性.
美国在科学和工程研究与创新方面的领导地位植根于国家独特的政府-大学-产业研发生态系统.
正如美国艺术与科学协会所写,"美国作为创新领导者的地位"依赖于"建立一个更强大的国家政府-大学-工业研究伙伴关系.
"126自2016年国家人工智能研发战略计划发布以来,政府已经扩大了这一愿景,即与学术界,工业界,国际合作伙伴和盟国以及其他非联邦实体合作,推动"对人工智能研发的持续投资,以实现人工智能的技术突破和相关技术,并将这些突破迅速转变为有助于美国经济和国家安全的能力.
"1在过去的几十年里,在拥有联邦资金和工业的大学进行的信息技术基础研究已经为国家经济带来了新的,数十亿美元的新领域.
127政府,大学和工业界的共同进步是相辅相成的,并导致了一个创新,充满活力的人工智能部门.
今天的许多人工智能系统都是如此由美国政府-大学-工业研发生态系统实现(见附文).
自2016年国家人工智能研发战略计划发布以来,人们更加重视公私伙伴关系的益处.
这些好处包括战略性地利用资源,包括设施,数据集和专业知识,以推动科学和工程创新;126恢复基础:研究在保护美国梦中的重要作用(美国艺术与科学学院,剑桥,马萨诸塞州,2014年);https://www.
amacad.
org/multimedia/pdfs/publications/researchpapersmonographs美国基础研究基金会.
127国家研究委员会计算机科学电信委员会,信息技术的持续创新(国家科学院出版社,华盛顿特区,2012年);https://doi.
org/10.
17226/13427.
推进国家的人工智能创新生态系统,涵盖政府,大学和行业深度卷积神经网络已被证明是植根于人工智能研究的关键创新.
虽然这种建模方法源自20世纪80年代后期的早期联邦投资,但当时神经网络没有足够的数据和足够的计算能力来进行准确的预测.
通过大数据的增加,当今数据密集型科学方法的结合,以及如何构建和优化网络的概念上的进步,神经网络已经重新成为提高AI模型准确性的有效方法.
近年来,学术界和私营部门之间的相互作用,包括政府资助,有助于降低语音识别系统的错误率,实现实时翻译等创新.
.
126同样,联邦在20世纪80年代和90年代对强化学习的投资-一种植根于行为心理学的方法,涉及学习将行为与期望的结果联系起来-导致了今天的深度学习系统.
通过跨部门的互动,计算机越来越像人类一样学习,没有明确的指导,强化学习推动了自动驾驶汽车和其他形式的自动化的进步,其中机器可以通过经验磨练技能.
强化学习是AlphaGo的关键技术,该计划击败了世界上最好的围棋选手,自2016年以来,该选手已经取得了越来越多的职业选手胜利.
126国家人工智能研发战略计划:2019–45–加速这些创新向实践的转变;并加强对下一代研究人员,技术人员和领导者的教育和培训.
政府-大学-产业研发合作伙伴关系为大学研究人员带来了工业所面临的紧迫的现实挑战,实现了"使用启发式研究";利用行业专业知识,加速将开放和已发表的研究成果转化为市场中可行的产品和服务,促进经济增长;通过将大学教师和学生与行业代表,行业环境和行业工作联系起来,增加研究和劳动力的能力(见附文).
126,128,129,130这些伙伴关系建立在联邦机构之间的联合参与的基础上,在机构任务交叉的领域实现协同增效.
国家也受益于联邦机构和国际资助者之间的关系,他们可以共同努力解决各个学科共同关心的关键挑战.
虽然公私伙伴关系有许多结构和机制,但参与的一些常见类别包括:1.
基于项目的单独协作.
这些努力使大学研究人员能够与其他部门(包括联邦机构,行业和国际组织)的人员进行接触,以确定和利用共同感兴趣领域的协同作用.
2.
促进开放,竞争前的基础研究的联合计划.
跨部门组织之间的直接合作伙伴关系可以为合作伙伴共同感兴趣的领域提供资金和支持开放,竞争前的基础研究.
一般而言,提供研究资源的非联邦合作伙伴获得Bayh-Dole法案赋予美国政府相同的知识产权.
1313.
合作部署和加强研究基础设施.
联邦机构,行业和国际组织之间的合作显着增强了开发新的和增强现有研究基础设施的潜力,从而使研究人员能够进行开创性的实验.
合作伙伴可以提供财务和/或实物支持,以开发和/或增强研究基础设施.
4.
合作以加强劳动力发展,包括扩大参与.
多部门合作伙伴关系为严谨,有吸引力和鼓舞人心的教学材料奠定了基础,这些材料可以增强STEM专业的劳动力发展和多样性.
在每一种情况下,为了所有人的利益而利用每个合作伙伴的优势对于取得成功至关重要.
128数学科学研究所报告,"伙伴关系:NSF/MPS与私人基金会合作研讨会",2015年;http://library.
msri.
org/msri/Partnerships.
pdf.
129计算社区联盟,"计算研究的未来:产业-学术合作",2016年;http://cra.
org/ccc/wp-content/uploads/sites/2/2016/06/15125-CCC-Industry-Whitepaper-v4-1.
pdf.
130计算社区联盟,"计算研究中不断发展的学术/行业关系:CCC发布的中期报告",2019年;https://www.
cccblog.
org/wp-content/uploads/2019/03/Industry-Interim-Report-w-footnotes.
pdf.
131https://history.
nih.
gov/research/downloads/PL96-517.
pdf国家人工智能研发战略计划:2019–46–人工智能研发的进步将受益于所有这些类型的公共私营合作伙伴关系.
伙伴关系可以促进开放的,竞争前的,基础的人工智能研发;增强对数据集,模型和高级计算能力等研究资源的访问;并促进政府,大学和行业之间的研究人员交流和/或联合任命,以分享AI研发专业知识.
伙伴关系也可以发布该人工智能研发具有固有的跨学科性质,需要计算机和信息科学,认知科学和心理学,经济学和博弈论,工程和控制理论,伦理学,语言学,数学和统计学以及哲学之间的融合,以推动未来AI的发展和评估系统公平,透明,负责,安全可靠.
联邦机构正在积极寻求公私合作以实现这些目标(见附文).
因此,联邦机构必须继续追求和加强人工智能研发中的公私合作伙伴关系,通过在政府,行业和学术界共同关心的领域中利用投资和专业知识来推动技术开发和经济增长.
在这样做的过程中,美国政府将利用一个独特的美国创新生态系统,通过新颖的信息技术在过去二十年中改变了国家经济和社会的各个方面.
.
127132https://www.
diu.
mil/133https://www.
dhs.
gov/science-and-technology/svip公私合作伙伴关系:近期的机构研发计划许多机构已经启动了公私合作伙伴关系,以支持人工智能研发:国防创新部门(DIU)132是一个国防部组织,寻求能够满足国防部需求的商业解决方案.
DIU反过来提供试验合同,其中可包括硬件,软件或其他独特服务.
如果成功,试点合同将导致公司与任何国防部实体之间的后续合同.
DIU的一个关键特征是试点和后续合同的快速发展.
NSF和人工智能合作伙伴关系是一个多元化,多利益相关方组织,旨在更好地了解AI的影响,正在合作共同支持AI的社会和技术层面的高风险,高回报研究.
15美国国土安全部科技理事会硅谷创新计划(SVIP)133希望利用全国乃至全球的商业研发创新生态系统来实现政府应用的技术.
SVIP采用简化的应用程序和音调流程;将政府,企业家和行业聚集在一起,寻找最前沿的解决方案;并共同投资并加速向市场的过渡.
卫生和人类服务部(HHS)试行了HealthTechSprint计划,该计划在其第一次迭代中称为"TopHealth",部分模仿人口普查局的机会项目.
这项工作为围绕双向数据链路的公私合作创建了一个灵活的框架.
它试验了用于迭代数据发布的新模型,用于人工智能培训和测试,并为公共-私人AI生态系统开发了一个自愿的激励框架.
HHS研究,创新和风险投资部门是负责准备和响应助理秘书办公室生物医学高级研究和发展管理局的一部分.
它负责监督加速器网络并正在招募一个非营利性合作伙伴,该合作伙伴可以与私人投资者合作,为创新技术和产品提供资金,以解决系统性健康安全挑战,人工智能应用程序是其中一个受关注的领域.
加速器将连接初创公司和其他企业的产品开发和业务支持服务.
国家人工智能研发战略计划:2019–47–缩略语空军科学研究局空军科学研究办公室嗳人工智能达帕国防高级研究计划局美国国土安全部国土安全部国防部国防部母鹿能源部点运输部FDA食品药品管理局FRVT人脸识别供应商测试GPS全球定位系统图形处理器图形处理单元GSA总务管理局卫生部卫生与人类服务部高性能混凝土高性能计算亚尔帕情报高级研究项目活动IEC国际电工委员会IEEE电气和电子工程师协会碰撞网络风险与信任政策与分析信息市场(DHS)异国际标准化组织它信息技术免疫球蛋白组机构间工作组毫升机器学习麦莱机器学习和人工智能(NSTC小组委员会)美国航空航天局国家航空和航天局军士NITRD国家协调办公室NDS自然驾驶研究(DOT)的NifA国家粮食和农业研究所(USDA)美国国立卫生研究院国立卫生研究院NIST国家标准与技术研究所硝石网络和信息技术研究与发展计划神经网络模型国家医学图书馆(NIH)NSF国家科学基金会NSTC国家科技评议会国家航空航天局国家电信和信息管理局奥德尼国家情报局局长办公室奥斯特普科技政策办公室R&D研究与开发射频干扰索取资料干科学,技术,工程和数学SVIP硅谷创新计划(DHS)特雷克文本检索会议美国农业部美国农业部佤族美国退伍军人事务部赛可解释的AI国家人工智能研发战略计划:2019–48–椅子国家科学技术委员会员工OSTP主任KelvinDroegemeierChloéKontos,NSTC执行董事人工智能专责委员会联合主席MichaelKratsios,技术政策总裁助理(白宫)法国A.
Córdova,NSF主任DARPA主任StevenWalker机器学习和人工智能小组委员会联合主席LynnetParker,OSTP人工智能助理总监美国国家科学基金会计算机信息科学与工程理事会(CISE)助理主任JamesKuroseNIST信息技术实验室主任CharlesRomine美国能源部科学办公室科学项目副主任StephenBinkley行政秘书费萨尔-德苏扎网络与信息技术研究与发展小组委员会联合主席KamieRoberts,NITRDNCO主任美国国家科学基金会CISE助理总监JamesKurose行政秘书NekeiaButler人工智能研究与开发机构间工作组联合主席IARPA国家情报总监(ODNI)项目经理JeffAlstott员工美国国家科学基金会信息与智能系统CISE部门主任HenryKautzNITRDNCO技术协调员FaisalD'SouzaJeffAlstott,IARPAGilAlterovitz,VASameerAntani,美国国立卫生研究院夏洛特贝尔,NIFA,美国农业部DanielClouse费萨尔-德苏扎金伯利弗格森-沃尔特,美国海军战略计划编写团队MichaelGarris,NIST欧文GiangdANi,NSF罗斯吉尔菲兰,OSTP特拉维斯厅,NTIA美国国家科学基金会梅根霍顿HenryKautz,NSFErinkNeNLY,DHSDavidKuehn,点国家人工智能研发战略计划:2019–49–JamesKurose,NSF杰姆斯劳顿,AFOSRStevenLee,DOEAaronMannes,DHS琳恩派克,OSTPDineshPatwardhanNIST的ElhamTabassi国家人工智能研发战略计划:2019年更新关于国家科学技术委员会国家科学技术委员会(NSTC)是行政部门在构成联邦研究和开发企业的各种实体之间协调科学和技术政策的主要手段.
NSTC的主要目标是确保科学和技术政策决策和计划符合总统的既定目标.
NSTC准备了旨在实现多个国家目标的联邦机构协调的研究和发展战略.
NSTC的工作由委员会组织,委员会负责监督关注科学和技术不同方面的小组委员会和工作组.
有关更多信息,请访问:https://www.
whitehouse.
gov/ostp/nstc.
关于科技政策办公室科学和技术政策办公室(OSTP)是根据1976年国家科学和技术政策,组织和优先事项法案设立的,为总统和总统执行办公室内的其他人提供科学,工程和技术方面的建议.
经济,国家安全,国土安全,健康,对外关系,环境,技术恢复和资源利用等方面.
OSTP领导机构间科学和技术政策协调工作,协助管理和预算办公室对预算中的联邦研究与开发(R&D)进行年度审查和分析,并作为总统的科学和技术分析和判断来源.
尊重联邦政府的主要政策,计划和计划.
有关更多信息,请访问:https://www.
whitehouse.
gov/ostp.
关于人工智能专责委员会人工智能特别委员会(AI)建议并协助NSTC提高与AI相关的联邦研发工作的整体效率和生产力,以确保美国在该领域的持续领导地位.
它涉及跨越机构界限的国家和国际政策事务,并为联邦人工智能研发活动提供机构间政策协调和发展的正式机制,包括与自治系统,生物识别,计算机视觉,人机交互,机器学习,自然相关的活动.
语言处理和机器人技术.
它还就机构间人工智能研发优先事项向总统执行办公室提供建议;致力于创建平衡和全面的AI研发计划和合作伙伴关系;利用跨部门和机构任务的联邦数据和计算资源;并支持技术,国家AI人力资源.
关于机器学习和人工智能小组委员会机器学习和人工智能(MLAI)小组委员会监督联邦政府,私营部门和国际机器学习(ML)和人工智能的最新技术,以监测重要技术里程碑的发展.
AI,协调联邦政府使用和促进有关ML和AI的知识和最佳实践的分享,并参与制定联邦MLAI研发优先事项.
MLAI小组委员会向技术委员会和AI专门委员会报告.
MLAI小组委员会还与人工智能研究与发展机构间工作组(见下文)协调AI任务.
关于网络与信息技术研究与发展小组委员会网络和信息技术研究与发展(NITRD)计划是Nation在联邦政府资助的计算,网络和软件领域的先驱信息技术(IT)工作的主要来源.
NITRD小组委员会指导多机构NITRD计划的工作,为确保美国技术领先地位和满足国家对先进IT的需求提供研发基础.
它向NSTC科学技术企业委员会报告.
小组委员会得到向其及其国家协调办公室报告的机构间工作组的支持.
有关更多信息,请访问:https://www.
nitrd.
gov/about/.
关于人工智能研究与开发跨部门工作组NITRD人工智能研发机构间工作组(IWG)协调人工智能的联邦研发;它还支持和协调人工智能专题委员会和NSTC机器学习和人工智能小组委员会的任务.
这项重要工作促进了美国在人工智能研发方面的领导地位和全球竞NITRD人工智能研发IWG牵头更新了这项国家人工智能研究与发展战略计划.
有关更多信息,请访问:https://www.
nitrd.
gov/groups/AI.
关于本文件本文件包括2016年国家AI研发战略计划的原始文本,其中包括2016年计划管理和机构间评估后的2019年更新以及社区对更新计划信息请求的回复.
2016年的战略被广泛认为是有效的,并且需要进行一些重新审核,并呼吁制定关于人工智能的私人-公共伙伴关系的新战略.
每个策略的顶部都插入了一个带阴影的标注框,以突出显示更新的命令和/或新的焦点区域.
2019年的更新增加了关于人工智能私人-公共伙伴关系的全新战略8.
版权信息本文件是美国政府的一项工作,属于公共领域(见17USC§105).
它可以自由分发,复制和翻译,并向OSTP确认;必须向OSTP提出使用任何图像的请求.
发表于2019年的美利坚合众国.
他"t'IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII白宫THENATIONALARTIFICIALINTELLIGENCERESEARCHANDDEVELOPMENTSTRATEGICPLAN:2019UPDATEAReportbytheSELECTCOMMITTEEONARTIFICIALINTELLIGENCEoftheNATIONALSCIENCE&TECHNOLOGYCOUNCILJUNE2019DearColleagues,InhisStateoftheUnionaddressonFebruary5,2019,PresidentTrumpstressedtheimportanceofensuringAmericanleadershipinthedevelopmentofemergingtechnologies,includingartificialintelligence(AI),thatmakeuptheIndustriesoftheFuture.
Reflectingthisimportance,onFebruary11,2019,PresidentTrumpsignedExecutiveOrder13859,whichestablishedtheAmericanArtificialIntelligenceInitiative.
ThisInitiativeisawhole-of-governmentapproachformaintainingAmericanleadershipinAIandensuringthatAIbenefitstheAmericanpeopleandreflectsourNation'svalues.
ThefirstdirectiveinthisExecutiveOrderisforFederalagenciestoprioritizeAIresearchanddevelopment(R&D)intheirannualbudgetingandplanningprocess.
TheattachedNationalAIR&DStrategicPlan:2019UpdatehighlightsthekeyprioritiesforFederalinvestmentinAIR&D.
Artificialintelligencepresentstremendousopportunitiesthatareleadingtobreakthroughsinimprovedhealthcare,saferandmoreefficienttransportation,personalizededucation,significantscientificdiscoveries,improvedmanufacturing,increasedagriculturalcropyields,betterweatherforecasting,andmuchmore.
Thesebenefitsarelargelyduetodecadesoflong-termFederalinvestmentsinfundamentalAIR&D,whichhaveledtonewtheoriesandapproachesforAIsystems,aswellasappliedresearchthatallowsthetranslationofAIintopracticalapplications.
ThelandscapeforAIR&Disbecomingincreasinglycomplex,duetothesignificantinvestmentsthatarebeingmadebyindustry,academia,andnonprofitorganizations.
Additionally,AIadvancementsareprogressingrapidly.
TheFederalGovernmentmustthereforecontinuallyreevaluateitsprioritiesforAIR&Dinvestments,toensurethatinvestmentscontinuetoadvancethecuttingedgeofthefieldandarenotunnecessarilyduplicativeofindustryinvestments.
InAugustof2018,theAdministrationdirectedtheSelectCommitteeonAItorefreshthe2016NationalAIR&DStrategicPlan.
ThisprocessbeganwiththeissuanceofaRequestforInformationtosolicitpublicinputonwaysthatthestrategyshouldberevisedorimproved.
TheresponsestothisRFI,aswellasanindependentagencyreview,informedthisupdatetotheStrategicPlan.
InthisStrategicPlan,eightstrategicprioritieshavebeenidentified.
Thefirstsevenstrategiescontinuefromthe2016Plan,reflectingthereaffirmationoftheimportanceofthesestrategiesbymultiplerespondentsfromthepublicandgovernment,withnocallstoremoveanyofthestrategies.
TheeighthstrategyisnewandfocusesontheincreasingimportanceofeffectivepartnershipsbetweentheFederalGovernmentandacademia,industry,othernon-Federalentities,andinternationalalliestogeneratetechnologicalbreakthroughsinAIandtorapidlytransitionthosebreakthroughsintocapabilities.
WhilethisPlandoesnotdefinespecificresearchagendasforFederalagencyinvestments,itdoesprovideanexpectationfortheoverallportfolioforFederalAIR&Dinvestments.
ThiscoordinatedFederalstrategyforAIR&DwillhelptheUnitedStatescontinuetoleadtheworldincutting-edgeadvancesinAIthatwillgrowoureconomy,increaseournationalsecurity,andimprovequalityoflife.
Sincerely,MichaelKratsiosDeputyAssistanttothePresidentforTechnologyPolicyJune21,2019THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–ii–TableofContentsExecutiveSummaryiiiIntroductiontothe2019NationalAIR&DStrategicPlan1AIR&DStrategy5Strategy1:MakeLong-TermInvestmentsinAIResearch.
72019Update:Sustaininglong-terminvestmentsinfundamentalAIresearch.
7Advancingdata-focusedmethodologiesforknowledgediscovery9EnhancingtheperceptualcapabilitiesofAIsystems.
9UnderstandingtheoreticalcapabilitiesandlimitationsofAI.
10Pursuingresearchongeneral-purposeartificialintelligence.
10DevelopingscalableAIsystems.
11Fosteringresearchonhuman-likeAI.
11Developingmorecapableandreliablerobots.
11AdvancinghardwareforimprovedAI12CreatingAIforimprovedhardware12Strategy2:DevelopEffectiveMethodsforHuman-AICollaboration142019Update:DevelopingAIsystemsthatcomplementandaugmenthumancapabilities,withincreasingfocusonthefutureofwork14Seekingnewalgorithmsforhuman-awareAI17DevelopingAItechniquesforhumanaugmentation17Developingtechniquesforvisualizationandhuman-AIinterfaces.
18Developingmoreeffectivelanguageprocessingsystems.
18Strategy3:UnderstandandAddresstheEthical,Legal,andSocietalImplicationsofAI.
192019Update:Addressingethical,legal,andsocietalconsiderationsinAI19Improvingfairness,transparency,andaccountabilitybydesign21BuildingethicalAI.
21DesigningarchitecturesforethicalAI.
21Strategy4:EnsuretheSafetyandSecurityofAISystems232019Update:CreatingrobustandtrustworthyAIsystems.
23Improvingexplainabilityandtransparency25Buildingtrust.
25Enhancingverificationandvalidation.
25Securingagainstattacks.
26Achievinglong-termAIsafetyandvalue-alignment26Strategy5:DevelopSharedPublicDatasetsandEnvironmentsforAITrainingandTesting.
272019Update:Increasingaccesstodatasetsandassociatedchallenges27DevelopingandmakingaccessibleawidevarietyofdatasetstomeettheneedsofadiversespectrumofAIinterestsandapplications29Makingtrainingandtestingresourcesresponsivetocommercialandpublicinterests.
30Developingopen-sourcesoftwarelibrariesandtoolkits.
30Strategy6:MeasureandEvaluateAITechnologiesthroughStandardsandBenchmarks.
322019Update:SupportingdevelopmentofAItechnicalstandardsandrelatedtools32DevelopingabroadspectrumofAIstandards.
33EstablishingAItechnologybenchmarks34IncreasingtheavailabilityofAItestbeds.
34EngagingtheAIcommunityinstandardsandbenchmarks.
35Strategy7:BetterUnderstandtheNationalAIR&DWorkforceNeeds.
372019Update:AdvancingtheAIR&Dworkforce,includingthoseworkingonAIsystemsandthoseworkingalongsidethem,tosustainU.
S.
leadership.
37Strategy8:ExpandPublic–PrivatePartnershipstoAccelerateAdvancesinAI.
40Abbreviations.
43THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–iii–ExecutiveSummaryArtificialintelligence(AI)holdstremendouspromisetobenefitnearlyallaspectsofsociety,includingtheeconomy,healthcare,security,thelaw,transportation,eventechnologyitself.
OnFebruary11,2019,thePresidentsignedExecutiveOrder13859,MaintainingAmericanLeadershipinArtificialIntelligence.
1ThisorderlaunchedtheAmericanAIInitiative,aconcertedefforttopromoteandprotectAItechnologyandinnovationintheUnitedStates.
TheInitiativeimplementsawhole-of-governmentstrategyincollaborationandengagementwiththeprivatesector,academia,thepublic,andlike-mindedinternationalpartners.
Amongotheractions,keydirectivesintheInitiativecallforFederalagenciestoprioritizeAIresearchanddevelopment(R&D)investments,enhanceaccesstohigh-qualitycyberinfrastructureanddata,ensurethattheNationleadsinthedevelopmentoftechnicalstandardsforAI,andprovideeducationandtrainingopportunitiestopreparetheAmericanworkforcefortheneweraofAI.
InsupportoftheAmericanAIInitiative,thisNationalAIR&DStrategicPlan:2019UpdatedefinesthepriorityareasforFederalinvestmentsinAIR&D.
This2019updatebuildsuponthefirstNationalAIR&DStrategicPlanreleasedin2016,accountingfornewresearch,technicalinnovations,andotherconsiderationsthathaveemergedoverthepastthreeyears.
ThisupdatehasbeendevelopedbyleadingAIresearchersandresearchadministratorsfromacrosstheFederalGovernment,withinputfromthebroadercivilsociety,includingfrommanyofAmerica'sleadingacademicresearchinstitutions,nonprofitorganizations,andprivatesectortechnologycompanies.
Feedbackfromthesekeystakeholdersaffirmedthecontinuedrelevanceofeachpartofthe2016StrategicPlanwhilealsocallingforgreaterattentiontomakingAItrustworthy,topartneringwiththeprivatesector,andotherimperatives.
TheNationalAIR&DStrategicPlan:2019UpdateestablishesasetofobjectivesforFederallyfundedAIresearch,identifyingthefollowingeightstrategicpriorities:Strategy1:Makelong-terminvestmentsinAIresearch.
PrioritizeinvestmentsinthenextgenerationofAIthatwilldrivediscoveryandinsightandenabletheUnitedStatestoremainaworldleaderinAI.
Strategy2:Developeffectivemethodsforhuman-AIcollaboration.
IncreaseunderstandingofhowtocreateAIsystemsthateffectivelycomplementandaugmenthumancapabilities.
Strategy3:Understandandaddresstheethical,legal,andsocietalimplicationsofAI.
ResearchAIsystemsthatincorporateethical,legal,andsocietalconcernsthroughtechnicalmechanisms.
Strategy4:EnsurethesafetyandsecurityofAIsystems.
AdvanceknowledgeofhowtodesignAIsystemsthatarereliable,dependable,safe,andtrustworthy.
Strategy5:DevelopsharedpublicdatasetsandenvironmentsforAItrainingandtesting.
Developandenableaccesstohigh-qualitydatasetsandenvironments,aswellastotestingandtrainingresources.
Strategy6:MeasureandevaluateAItechnologiesthroughstandardsandbenchmarks.
DevelopabroadspectrumofevaluativetechniquesforAI,includingtechnicalstandardsandbenchmarks.
Strategy7:BetterunderstandthenationalAIR&Dworkforceneeds.
ImproveopportunitiesforR&DworkforcedevelopmenttostrategicallyfosteranAI-readyworkforce.
Strategy8:Expandpublic-privatepartnershipstoaccelerateadvancesinAI.
PromoteopportunitiesforsustainedinvestmentinAIR&Dandfortransitioningadvancesintopracticalcapabilities,incollaborationwithacademia,industry,internationalpartners,andothernon-Federalentities.
1https://www.
whitehouse.
gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–1–Introductiontothe2019NationalAIR&DStrategicPlanArtificialintelligenceenablescomputersandotherautomatedsystemstoperformtasksthathavehistoricallyrequiredhumancognitionandwhatwetypicallyconsiderhumandecision-makingabilities.
Overthepastseveraldecades,AIhasadvancedtremendouslyandtodaypromisesbetter,moreaccuratehealthcare;enhancednationalsecurity;improvedtransportation;andmoreeffectiveeducation,tonamejustafewbenefits.
Increasedcomputingpower,theavailabilityoflargedatasetsandstreamingdata,andalgorithmicadvancesinmachinelearning(ML)havemadeitpossibleforAIdevelopmenttocreatenewsectorsoftheeconomyandrevitalizeindustries.
AsmoreindustriesadoptAI'sfundamentaltechnologies,thefieldwillcontinuetodriveprofoundeconomicimpactandquality-of-lifeimprovementsworldwide.
TheseadvancementshavebeendrivenprimarilybyFederalinvestmentsinAIR&D,theexpertiseofAmerica'sunsurpassedR&Dinstitutions,andthecollectivecreativityofmanyofAmerica'smostvisionarytechnologycompaniesandentrepreneurs.
In2016theFederalGovernmentpublishedfirstNationalAIR&DStrategicPlan,recognizingAI'stremendouspromiseandneedforcontinuedadvancement.
ItwasdevelopedtoguidetheNationinourAIR&Dinvestments,provideastrategicframeworkforimprovingandleveragingAmerica'sAIcapabilities,andensurethatthosecapabilitiesproduceprosperity,security,andimprovedqualityoflifefortheAmericanpeopleforyearstocome.
ThePlandefinedseveralkeyareasofpriorityfocusfortheFederalagenciesthatinvestinAI.
Thesefocusareas,orstrategies,include:continuedlong-terminvestmentsinAI;effectivemethodsforhuman-AIcollaboration;understandingandaddressingtheethical,legal,andsocietalimplicationsforAI;ensuringthesafetyandsecurityofAI;developingsharedpublicdatasetsandenvironmentsforAItrainingandtesting;measuringandevaluatingAItechnologiesthroughstandardsandbenchmarks;andbetterunderstandingtheNation'sAIR&D2https://www.
nitrd.
gov/news/RFI-National-AI-Strategic-Plan.
aspx3https://www.
nitrd.
gov/nitrdgroups/index.
phptitle=AI-RFI-Responses-20182019UpdateRFIresponsesinformthe2019NationalAIR&DStrategicPlanInSeptember2018,theNationalCoordinationOfficeforNetworkingandInformationTechnologyResearchandDevelopmentissuedaRequestforInformation(RFI)2onbehalfoftheSelectCommitteeonArtificialIntelligence,requestinginputfromallinterestedpartiesonthe2016NationalArtificialIntelligenceResearchandDevelopmentStrategicPlan.
Nearly50responsesweresubmittedbyresearchers,researchorganizations,professionalsocieties,civilsocietyorganizations,andindividuals;theseresponsesareavailableonline.
3Manyoftheresponsesreaffirmedtheanalysis,organization,andapproachoutlinedinthe2016NationalAIR&DStrategicPlan.
AsignificantnumberofresponsesnotedtheimportanceofinvestingintheapplicationofAIinareassuchasmanufacturingandsupplychains;healthcare;medicalimaging;meteorology,hydrology,climatology,andrelatedareas;cybersecurity;education;data-intensivephysicalsciencessuchashigh-energyphysics;andtransportation.
ThisinterestintranslationalapplicationsofAItechnologieshascertainlyincreasedsincethereleaseofthe2016NationalAIR&DStrategicPlan.
OthercommonthemesechoedintheRFIresponsesweretheimportanceofdevelopingtrustworthyAIsystems,includingfairness,ethics,accountability,andtransparencyofAIsystems;curatedandaccessibledatasets;workforceconsiderations;andpublic-privatepartnershipsforfurtheringAIR&D.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–2–workforceneeds.
Thatworkwasprescient:today,countriesaroundtheworldhavefollowedsuitandhaveissuedtheirownversionsofthisplan.
InthethreeyearssincetheNationalAIR&DStrategicPlanwasproduced,newresearch,technicalinnovations,andreal-worlddeploymentshaveprogressedrapidly.
TheAdministrationinitiatedthis2019updatetotheNationalAIR&DStrategicPlantoaddresstheseadvancements,includingarapidlyevolvinginternationalAIlandscape.
Notably,this2019UpdatetotheNationalAIR&DStrategicPlanis,bydesign,solelyconcernedwithaddressingtheresearchanddevelopmentprioritiesassociatedwithadvancingAItechnologies.
ItdoesnotdescribeorrecommendpolicyorregulatoryactionsrelatedtothegovernanceordeploymentofAI,althoughAIR&Dwillcertainlyinformthedevelopmentofreasonablepolicyandregulatoryframeworks.
AIasanAdministrationPrioritySince2017,theAdministrationhasaddressedtheimportanceofAIR&DbyemphasizingitsroleforAmerica'sfutureacrossmultiplemajorpolicydocuments,includingtheNationalSecurityStrategy,4theNationalDefenseStrategy,5andtheFY2020R&DBudgetPrioritiesMemo.
6InMay2018,theOfficeofScienceandTechnologyPolicy(OSTP)hostedtheWhiteHouseSummitonArtificialIntelligenceforAmericanIndustrytobegindiscussingthepromiseofAIandthepoliciesneededtorealizethatpromisefortheAmericanpeopleandmaintainU.
S.
leadershipintheageofAI.
TheSummitconvenedover100seniorgovernmentofficials,technicalexpertsfromtopacademicinstitutions,headsofindustrialresearchlaboratories,andAmericanbusinessleaders.
InhisStateoftheUnionaddressonFebruary5,2019,PresidentTrumpstressedtheimportanceofensuringAmericanleadershipinthedevelopmentofemergingtechnologies,includingAI,thatmakeuptheIndustriesoftheFuture.
OnFebruary11,2019,thePresidentsignedExecutiveOrder13859,MaintainingAmericanLeadershipinArtificialIntelligence.
7ThisorderlaunchedtheAmericanAIInitiative,aconcertedefforttopromoteandprotectAItechnologyandinnovationintheUnitedStates.
TheInitiativeimplementsawhole-of-governmentstrategyincollaborationandengagementwiththeprivatesector,academia,thepublic,andlike-mindedinternationalpartners.
Amongotheractions,keydirectivesintheInitiativecallforFederalagenciestoprioritizeAIR&Dinvestments,enhanceaccesstohigh-qualitycyberinfrastructureanddata,ensurethattheNationleadsinthedevelopmentoftechnicalstandardsforAI,andprovideeducationandtrainingopportunitiestopreparetheAmericanworkforcefortheneweraofAI.
Developmentofthe2019UpdatetotheNationalAIR&DStrategicPlanThe2016NationalAIR&DStrategicPlanrecommendedthatthemanyFederalagenciestaskedwithadvancingoradoptingAIcollaboratetoidentifycriticalR&DopportunitiesandsupporteffectivecoordinationofFederalAIR&Dactivities,bothintramuralandextramuralresearch.
ReflectingtheAdministration'sprioritizationofAI,theNationalScienceandTechnologyCouncil(NSTC)hasestablishedanewframeworktoimplementthisrecommendation,consistingofthreeuniqueNSTCsubgroupsmadeupofmembersfromacrosstheFederalR&Dagenciestocover(1)seniorleadership4https://www.
whitehouse.
gov/wp-content/uploads/2017/12/NSS-Final-12-18-2017-0905.
pdf5https://dod.
defense.
gov/Portals/1/Documents/pubs/2018-National-Defense-Strategy-Summary.
pdf6https://www.
whitehouse.
gov/wp-content/uploads/2018/07/M-18-22.
pdf7https://www.
whitehouse.
gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–3–andstrategicvision,(2)operationalplanningandtacticalimplementation,and(3)researchandtechnicalexpertise.
Thesesubgroupsare:TheSelectCommitteeonAI,8consistingoftheheadsofdepartmentsandagenciesprincipallyresponsibleforthegovernment'sAIR&D,advisestheAdministrationoninteragencyAIR&Dpriorities;considersthecreationofFederalpartnershipswithindustryandacademia;establishesstructurestoimprovegovernmentplanningandcoordinationofAIR&D;identifiesopportunitiestoleverageFederaldataandcomputationalresourcestosupportournationalAIR&Decosystem;andsupportsthegrowthofatechnical,nationalAIworkforce.
TheNSTCSubcommitteeonMachineLearningandArtificialIntelligence(MLAI),consistingofagencyAIleadersandadministrators,servesastheoperationalandimplementationarmoftheSelectCommittee,responsibleforfulfillingtaskingfromtheSelectCommittee;creatingandmaintainingtheNationalAIR&DStrategicPlan;identifyingandaddressingimportantpolicyissuesrelatedtoAIresearch,testing,standards,education,implementation,outreach,andrelatedareas;andrelatedactivities.
TheAIR&DInteragencyWorkingGroup,operatingundertheNSTC'sNetworkingandInformationTechnologyR&D(NITRD)SubcommitteeandconsistingofresearchprogrammanagersandtechnicalexpertsfromacrosstheFederalGovernment,reportstotheMLAISubcommittee;helpscoordinateinteragencyAIR&Dprogrammaticefforts;servesastheinteragencyAIR&Dcommunityofpractice;andreportsgovernment-wideAIR&DspendingthroughtheNITRDSubcommittee'sannualSupplementtothePresident'sBudget.
InSeptember2018,theSelectCommitteeinitiatedanupdatetothe2016StrategicPlan,beginningwithanRFIseekingbroadcommunityinputonwhetherandhowthesevenstrategiesofthe2016NationalAIR&DStrategicPlanmeritedrevisionorreplacement(seesidebar).
Independently,FederaldepartmentsandagenciesperformingorfundingAIR&Dundertooktheirownassessments.
AnOverviewofthe2019Updatetothe2016NationalAIR&DStrategicPlanTogether,theSelectCommitteeonAI,theNSTCSubcommitteeonMachineLearningandAI,andtheAIR&DInteragencyWorkingGroupofNITRDreviewedtheinputregardingtheNationalAIR&DStrategicPlan.
Eachoftheoriginalsevenfocusareasorstrategiesofthe2016Planwasreaffirmedbymultiplerespondentsfromthepublicandgovernment,withnocallstoremoveanyonestrategy.
Thesestrategies,updatedinthis2019UpdatetotheStrategicPlantoreflectthecurrentstateoftheart,are:Strategy1:Makelong-terminvestmentsinAIresearch;Strategy2:Developeffectivemethodsforhuman-AIcollaboration;Strategy3:Understandandaddresstheethical,legal,andsocietalimplicationsofAI;Strategy4:EnsurethesafetyandsecurityofAIsystems;Strategy5:DevelopsharedpublicdatasetsandenvironmentsforAItrainingandtesting;Strategy6:MeasureandevaluateAItechnologiesthroughstandardsandbenchmarks;andStrategy7:BetterunderstandthenationalAIR&Dworkforceneeds.
8https://www.
whitehouse.
gov/wp-content/uploads/2018/05/Summary-Report-of-White-House-AI-Summit.
pdfTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–4–ManyresponsestotheRFIcalledforgreaterFederalGovernmentR&Dengagementwiththeprivatesector,giventhefastriseofprivatelyfundedAIR&D,andtherapidadoptionofAIbyindustry.
Asaresult,the2019Updateincorporatesanew,eighthstrategy:Strategy8:Expandpublic-privatepartnershipstoaccelerateadvancesinAI.
FeedbackfromthepublicandFederalagenciesidentifiedanumberofspecificchallengestofurtherAIdevelopmentandadoption.
Thesechallenges,manyofwhichcutacrossmultipleagencies,provideenhancedinsightintowaysthatthisNationalAIR&DStrategicPlancanguidethecourseofAIR&DinAmerica,andmanycloselyrelatetothethemesaddressedinthe2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligence.
Examplesincludethefollowing:Researchatthefrontiers.
Eventhoughmachinelearninghasbroughtphenomenalnewcapabilitiesinthepastseveralyears,continuedresearchisneededtofurtherpushthefrontiersofML,aswellastodevelopadditionalapproachestothetoughtechnicalchallengesofAI(Strategy1).
Positiveimpact.
AsAIcapabilitiesgrow,theUnitedStatesmustplaceincreasedemphasisondevelopingnewmethodstoensurethatAI'simpactsarerobustlypositiveintothefuture(Strategies1,3,and4).
Trustandexplainability.
TrulytrustworthyAIrequiresexplainableAI,especiallyasAIsystemsgrowinscaleandcomplexity;thisrequiresacomprehensiveunderstandingoftheAIsystembythehumanuserandthehumandesigner(Strategies1,2,3,4,and6).
Safetyandsecurity.
ResearchersmustdevisemethodstokeepAIsystemsandthedatatheyusesecuresothattheNationcanleveragetheopportunitiesaffordedbythistechnologywhilealsomaintainingconfidentialityandsafety(Strategies4,5,and6).
Technicalstandards.
AstheNationdevelopstechniquestoexpandbothAIabilitiesandassurance,itmusttestandbenchmarkthem;whenthetechniquesareready,theyshouldbeturnedintotechnicalstandardsfortheworld(Strategy6).
Workforcecapability.
AccomplishingthesegoalswillrequiregrowingaskilledAIR&Dworkforcethatiscurrentlylimitedandinhighdemand;theUnitedStatesmustbecreativeandboldintrainingandacquiringtheskilledworkforceitneedstoleadtheworldinAIresearchandapplications(Strategy7).
Partnerships.
AdvancesinAIR&DincreasinglyrequireeffectivepartnershipsbetweentheFederalGovernmentandacademia,industry,andothernon-FederalentitiestogeneratetechnologicalbreakthroughsinAIandtorapidlytransitionthosebreakthroughsintocapabilities(Strategy8).
Cooperationwithallies.
Additionally,thePlanrecognizestheimportanceofinternationalcooperationforsuccessfulimplementationofthesegoals,whileprotectingtheAmericanAIR&Denterprisefromstrategiccompetitorsandadversarialnations.
Structureofthis2019Updatetothe2016NationalAIR&DStrategicPlanThisupdatedNationalAIR&DStrategicPlanincorporatestheoriginaltextfromthe2016version,includingthefollowingsectiononR&DStrategy(exceptforminoredits)andtheoriginal2016wordingofthefirstsevenstrategies.
Foreachstrategy,2019updatestothe2016NationalR&DStrategicPlanareprovidedinshadedboxesatthetopoftheoriginalsevenstrategies;thesehighlightupdatedimperativesand/ornewfocusareasforthestrategies.
Textbelowtheshadedboxesisasitoriginallyappearedinthe2016NationalAIR&DStrategicPlan,providingobservationsandcontextthatremainimportanttoday(notethatsomeoftheoriginaldetailsmayhavebecomeoutofdateintheinterveningperiod).
Inaddition,asnotedpreviously,aneweighthstrategyisaddedinthis2019Update,onexpandingpublic-privatepartnershipsinAIR&D.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–5–AIR&DStrategyTheresearchprioritiesoutlinedinthisAIR&DStrategicPlanfocusonareasthatindustryisunlikelytoaddressontheirown,andthus,areasthataremostlikelytobenefitfromFederalinvestment.
TheseprioritiescutacrossallofAItoincludeneedscommontotheAIsub-fieldsofperception,automatedreasoning/planning,cognitivesystems,machinelearning,naturallanguageprocessing,robotics,andrelatedfields.
BecauseofthebreadthofAI,theseprioritiesspantheentirefield,ratherthanonlyfocusingonindividualresearchchallengesspecifictoeachsub-domain.
Toimplementtheplan,detailedroadmapsshouldbedevelopedthataddressthecapabilitygapsconsistentwiththeplan.
OneofthemostimportantFederalresearchpriorities,outlinedinStrategy1,isforsustainedlong-termresearchinAItodrivediscoveryandinsight.
ManyoftheinvestmentsbytheU.
S.
FederalGovernmentinhigh-risk,high-reward9fundamentalresearchhaveledtorevolutionarytechnologicaladvanceswedependontoday,includingtheInternet,GPS,smartphonespeechrecognition,heartmonitors,solarpanels,advancedbatteries,cancertherapies,andmuch,muchmore.
ThepromiseofAItouchesnearlyeveryaspectofsocietyandhasthepotentialforsignificantpositivesocietalandeconomicbenefits.
Thus,tomaintainaworldleadershippositioninthisarea,theUnitedStatesmustfocusitsinvestmentsonhigh-priorityfundamentalandlong-termAIresearch.
ManyAItechnologieswillworkwithandalongsidehumans,thusleadingtoimportantchallengesinhowtobestcreateAIsystemsthatworkwithpeopleinintuitiveandhelpfulways.
10ThewallsbetweenhumansandAIsystemsareslowlybeginningtoerode,withAIsystemsaugmentingandenhancinghumancapabilities.
Fundamentalresearchisneededtodevelopeffectivemethodsforhuman-AIinteractionandcollaboration,asoutlinedinStrategy2.
AIadvancementsareprovidingmanypositivebenefitstosocietyandareincreasingU.
S.
nationalcompetitiveness.
11However,aswithmosttransformativetechnologies,AIpresentssomesocietalrisksinseveralareas,fromjobsandtheeconomytosafety,ethical,andlegalquestions.
Thus,asAIscienceandtechnologydevelop,theFederalGovernmentmustalsoinvestinresearchtobetterunderstandwhattheimplicationsareforAIforalltheserealms,andtoaddresstheseimplicationsbydevelopingAIsystemsthatalignwithethical,legal,andsocietalgoals,asoutlinedinStrategy3.
AcriticalgapincurrentAItechnologyisalackofmethodologiestoensurethesafetyandpredictableperformanceofAIsystems.
EnsuringthesafetyofAIsystemsisachallengebecauseoftheunusualcomplexityandevolvingnatureofthesesystems.
Severalresearchprioritiesaddressthissafetychallenge.
First,Strategy4emphasizestheneedforexplainableandtransparentsystemsthataretrustedbytheirusers,performinamannerthatisacceptabletotheusers,andcanbeguaranteedtoactastheuserintended.
ThepotentialcapabilitiesandcomplexityofAIsystems,combinedwiththewealthofpossibleinteractionswithhumanusersandtheenvironment,makesitcriticallyimportanttoinvestinresearchthatincreasesthesecurityandcontrolofAItechnologies.
Strategy5callsontheFederalGovernmenttoinvestinsharedpublicdatasetsforAItrainingandtestingtoadvancetheprogressofAIresearchandtoenableamoreeffectivecomparisonofalternativesolutions.
Strategy6discusseshowstandardsandbenchmarkscanfocusR&Dtodefineprogress,closegaps,anddriveinnovativesolutionsforspecificproblemsandchallenges.
Standardsandbenchmarksare9"High-risk,high-reward"researchreferstovisionaryresearchthatisintellectuallychallengingbuthasthepotentialtomakedeeplypositive,transformativeimpactsonthefieldofstudy.
10See2016ReportoftheOneHundredYearStudyonArtificialIntelligence,whichfocusesontheanticipatedusesandimpactsofAIintheyear2030;https://ai100.
stanford.
edu/2016-report.
11J.
Furman,"IsThisTimeDifferentTheOpportunitiesandChallengesofArtificialIntelligence,"CouncilofEconomicAdvisorsremarks,NewYorkUniversity:AINowSymposium,July7,2016.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–6–essentialformeasuringandevaluatingAIsystemsandensuringthatAItechnologiesmeetcriticalobjectivesforfunctionalityandinteroperability.
Finally,thegrowingprevalenceofAItechnologiesacrossallsectorsofsocietycreatesnewpressuresforAIR&Dexperts.
OpportunitiesaboundforcoreAIscientistsandengineerswithadeepunderstandingofthetechnologywhocangeneratenewideasforadvancingtheboundariesofknowledgeinthefield.
TheNationshouldtakeactiontoensureasufficientpipelineofAI-capabletalent.
Strategy7addressesthischallenge.
Figure1(updatedinthis2019versionofthePlan)providesagraphicalillustrationoftheoverallorganizationofthisAIR&DStrategicPlan.
Acrossthebottomrowofboxesarethecrosscutting,underlyingfoundationsthataffectthedevelopmentofallAIsystems;thesefoundationsaredescribedinStrategies3-7andthenewStrategy8.
Thenextlayerhigher(middlerowofboxes)includesmanyareasofresearchthatareneededtoadvanceAI.
TheseR&Dareas(includinguse-inspiredbasicresearch)areoutlinedinStrategies1-2.
12AcrossthetoprowofboxesinthegraphicareexamplesofapplicationsthatareexpectedtobenefitfromadvancesinAI.
Together,thesecomponentsoftheAIR&DStrategicPlandefineahigh-levelframeworkforFederalinvestmentsthatcanleadtoimpactfuladvancesinthefieldandpositivesocietalbenefits.
Figure1.
OrganizationoftheAIR&DStrategicPlan(2019update,toincludeStrategy8).
AcombinationofcrosscuttingR&Dfoundations(inthelowerrow)areimportantforallAIresearch.
ManyAIR&Dareas(inthemiddlerow)canbuilduponthesecrosscuttingfoundationstoimpactawidearrayofsocietalapplications(inthetoprow).
ThenumbersinbracketsindicatethenumberoftheStrategyinthisplanthatfurtherdevelopseachtopic.
Theorderingofthesestrategiesdoesnotindicateapriorityofimportance.
12Throughoutthisdocument,"basicresearch"includesbothpurebasicresearchanduse-inspiredbasicresearch—theso-calledPasteur'sQuadrantdefinedbyDonaldStokesinhis1997bookofthesamename—referringtobasicresearchthathasuseforsocietyinmind.
Forexample,thefundamentalNIHinvestmentsinITareoftencalleduse-inspiredbasicresearch.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–7–Strategy1:MakeLong-TermInvestmentsinAIResearch2019UpdateSustaininglong-terminvestmentsinfundamentalAIresearchSincethereleaseofthe2016NationalAIR&DStrategicPlan,powerfulnewcapabilities,primarilyMLapplicationstowell-definedtasks,havecontinuedtoemerge.
Thesecapabilitieshavedemonstratedimpactsinadiversearrayofapplications,suchasclassifyinggeneticsequences,20,21managinglimitedwirelessspectrumresources,22interpretingmedicalimages,23andgradingcancers.
24Theserapidadvancesrequireddecadesofresearchforthetechnologiesandapplicationstomature.
25TomaintainthisprogressinMLtoachieveadvancementsinotherareasofAI,andtostrivetowardthelong-termgoalofgeneral-purposeAI,theFederalGovernmentmustcontinuetofosterlong-term,fundamentalresearchinMLandAI.
Thisresearchwillgiverisetotransformationaltechnologiesand,inturn,breakthroughsacrossallsectorsofsociety.
Muchofthecurrentprogressinthefieldhasbeeninspecialized,well-definedtasksoftendrivenbystatisticalML,suchasclassification,recognition,andregression(i.
e.
,"narrowAIsystems").
SurveysoftheLong-term,fundamentalAIresearch:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,anumberofagencieshaveinitiatedAIR&DprogramsforStrategy1:NSFhascontinuedtofundfoundationalresearchinAI,spanningML,reasoningandrepresentation,computervision,computationalneuroscience,speechandlanguage,robotics,andmulti-agentsystems.
NSFhaslaunchednewjointfundingopportunitieswithotheragencies—notablywithDARPAintheareaofhigh-performance,energy-efficienthardwareforreal-timeML13andwithUSDA-NIFAonAIforagriculturalscience14—andwithindustry.
15,16Inaddition,NSF'sHarnessingtheDataRevolutionBigIdea17supportsresearchonthefoundationsofdatascience,whichwillserveasadriveroffutureMLandAIsystems.
DARPAannouncedinSeptember2018amultiyearinvestmentinnewandexistingprogramscalledthe"AINext"campaign.
18KeycampaignareasincludeimprovingtherobustnessandreliabilityofAIsystems;enhancingthesecurityandresiliencyofML/AItechnologies;reducingpower,data,andperformanceinefficiencies;andpioneeringthenextgenerationofAIalgorithmsandapplications,suchasexplainabilityandcommonsensereasoning.
TheNIHStrategicPlanforDataScience19ofSeptember2018aimstoadvanceaccesstodatasciencetechnologyandML/AIcapabilityforthebiomedicalresearchcommunitytowarddata-drivenhealthcareresearch.
13https://www.
nsf.
gov/funding/pgm_summ.
jsppims_id=505640&org=NSF14https://www.
nsf.
gov/pubs/2019/nsf19051/nsf19051.
jsp15https://www.
nsf.
gov/pubs/2019/nsf19018/nsf19018.
jsp16https://www.
nsf.
gov/funding/pgm_summ.
jsppims_id=50565117https://www.
nsf.
gov/cise/harnessingdata/18https://www.
darpa.
mil/work-with-us/ai-next-campaign19https://datascience.
nih.
gov/strategicplan20https://ai.
googleblog.
com/2017/12/deepvariant-highly-accurate-genomes.
html21https://irp.
nih.
gov/catalyst/v26i4/machine-learning22https://www.
spectrumcollaborationchallenge.
com/23https://news-medical.
net/news/20190417/Workshop-explores-the-future-of-artificial-intelligence-in-medical-imaging.
aspx24https://www.
nature.
com/articles/nature2105625https://www.
nitrd.
gov/rfi/ai/2018/AI-RFI-Response-2018-Yolanda-Gil-AAAI.
pdfTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–8–fieldhavenotedthatlong-terminvestmentsinfundamentalresearchareneededtocontinuebuildingontheseadvancesinML.
Further,parallelsustainedeffortsarerequiredtofullyrealizethevisionof"general-purposeAI"—systemsthatexhibittheflexibilityandversatilityofhumanintelligenceinabroadrangeofcognitivedomains.
26,27,28,29EmphasisisneededonthedevelopmentoffurtherMLcapabilitiestointeractivelyandpersistentlylearn,theconnectionbetweenperceptionandattention,andtheincorporationoflearnedmodelsintocomprehensivereasoningarchitectures.
30BeyondML,criticalresearchisalsoneededinothercoreareasofAI,includingincommonsensereasoningandproblemsolving,probabilisticreasoning,combinatorialoptimization,knowledgerepresentation,planningandscheduling,naturallanguageprocessing,decisionmaking,andhuman-machineinteraction.
Advancesintheseareaswillinturnenablecollaborativeroboticsandsharedandfullyautonomoussystems(seeStrategy2).
Thegrandchallengeofunderstandinghumanintelligencerequiressignificantinvestmentsinsharedresourcesandinfrastructure.
25BroadconsensusexistsforfoundationalinvestmentsindriversofMLandAIaswell,includingdataprovenanceandquality,novelsoftwareandhardwareparadigmsandplatforms,andthesecurityofAIsystems.
31,32Forexample,asAIsoftwareperformsincreasinglycomplexfunctionsinallaspectsofdailylifeandallsectorsoftheeconomy,existingsoftwaredevelopmentparadigmswillneedtoevolvetomeetsoftwareproductivity,quality,andsustainabilityrequirements.
RecentFederalinvestmentshaveprioritizedtheseareasoffundamentalMLandAIresearch(seesidebar)aswellastheuseofMLandAIacrossnumerousapplicationsectors,includingdefense,security,energy,transportation,health,agriculture,andtelecommunications.
Ultimately,AItechnologiesarecriticalforaddressingarangeoflong-termchallenges,suchasconstructingadvancedhealthcaresystems,arobustintelligenttransportationsystem,andresilientenergyandtelecommunicationnetworks.
ForAIapplicationstobecomewidespread,theymustbeexplainableandunderstandable(seeStrategy3).
Thesechallengesareparticularlysalientforfosteringcollaborativehuman-AIrelationships(seeStrategy2).
Today,theabilitytounderstandandanalyzethedecisionsofAIsystemsandmeasuretheiraccuracy,reliability,andreproducibilityislimited.
SustainedR&DinvestmentsareneededtoadvancetrustinAIsystemstoensuretheymeetsociety'sneedsandadequatelyaddressrequirementsforrobustness,fairness,explainability,andsecurity.
Along-termcommitmenttoAIR&DisessentialtocontinueandexpandcurrenttechnicaladvancesandmorebroadlyensurethatAIenrichesthehumanexperience.
Indeed,the2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligencenotes:HeadsofimplementingagenciesthatalsoperformorfundR&D(AIR&Dagencies),shallconsiderAIasanagencyR&Dpriority,asappropriatetotheirrespectiveagencies'missions…HeadsofsuchagenciesshalltakethispriorityintoaccountwhendevelopingbudgetproposalsandplanningfortheuseoffundsinFiscalYear2020andinfutureyears.
HeadsoftheseagenciesshallalsoconsiderappropriateadministrativeactionstoincreasefocusonAIfor2019.
26https://ai100.
stanford.
edu/sites/g/files/sbiybj9861/f/ai100report10032016fnl_singles.
pdf27http://cdn.
aiindex.
org/2018/AI%20Index%202018%20Annual%20Report.
pdf28https://cra.
org/ccc/visioning/visioning-activities/2018-activities/artificial-intelligence-roadmap/29https://www.
microsoft.
com/en-us/research/research-area/artificial-intelligence/30https://cra.
org/ccc/events/artificial-intelligence-roadmap-workshop-3-learning-and-robotics/31https://cra.
org/ccc/wp-content/uploads/sites/2/2016/04/AI-for-Social-Good-Workshop-Report.
pdf32https://openai.
com/blog/ai-and-compute/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–9–AIresearchinvestmentsareneededinareaswithpotentiallong-termpayoffs.
Whileanimportantcomponentoflong-termresearchisincrementalresearchwithpredictableoutcomes,long-termsustainedinvestmentsinhigh-riskresearchcanleadtohigh-rewardpayoffs.
Thesepayoffscanbeseenin5years,10years,ormore.
A2012NationalResearchCouncilreportemphasizesthecriticalroleofFederalinvestmentsinlong-termresearch,noting"thelong,unpredictableincubationperiod—requiringsteadyworkandfunding—betweeninitialexplorationandcommercialdeployment.
"33Itfurthernotesthat"thetimefromfirstconcepttosuccessfulmarketisoftenmeasuredindecades.
"Well-documentedexamplesofsustainedfundamentalresearcheffortsthatledtohigh-rewardpayoffsincludetheWorldWideWebanddeeplearning.
Inbothcases,thebasicfoundationsbeganinthe1960s;itwasonlyafter30+yearsofcontinuedresearcheffortsthattheseideasmaterializedintothetransformativetechnologieswitnessedtodayinmanycategoriesofAI.
Thefollowingsubsectionshighlightsomeoftheseareas.
AdditionalcategoriesofimportantAIresearcharediscussedinStrategies2through6.
Advancingdata-focusedmethodologiesforknowledgediscoveryAsdiscussedinthe2016FederalBigDataResearchandDevelopmentStrategicPlan,34manyfundamentalnewtoolsandtechnologiesareneededtoachieveintelligentdataunderstandingandknowledgediscovery.
Furtherprogressisneededinthedevelopmentofmoreadvancedmachinelearningalgorithmsthatcanidentifyalltheusefulinformationhiddeninbigdata.
Manyopenresearchquestionsrevolvearoundthecreationanduseofdata,includingitsveracityandappropriatenessforAIsystemtraining.
Theveracityofdataisparticularlychallengingwhendealingwithvastamountsofdata,makingitdifficultforhumanstoassessandextractknowledgefromit.
Whilemuchresearchhasdealtwithveracitythroughdataqualityassurancemethodstoperformdatacleaningandknowledgediscovery,furtherstudyisneededtoimprovetheefficiencyofdatacleaningtechniques,tocreatemethodsfordiscoveringinconsistenciesandanomaliesinthedata,andtodevelopapproachesforincorporatinghumanfeedback.
Researchersneedtoexplorenewmethodstoenabledataandassociatedmetadatatobeminedsimultaneously.
ManyAIapplicationsareinterdisciplinaryinnatureandmakeuseofheterogeneousdata.
Furtherinvestigationofmultimodalitymachinelearningisneededtoenableknowledgediscoveryfromawidevarietyofdifferenttypesofdata(e.
g.
,discrete,continuous,text,spatial,temporal,spatio-temporal,graphs).
AIinvestigatorsmustdeterminetheamountofdataneededfortrainingandtoproperlyaddresslarge-scaleversuslong-taildataneeds.
Theymustalsodeterminehowtoidentifyandprocessrareeventsbeyondpurelystatisticalapproaches;toworkwithknowledgesources(i.
e.
,anytypeofinformationthatexplainstheworld,suchasknowledgeofthelawofgravityorofsocialnorms)aswellasdatasources,integratingmodelsandontologiesinthelearningprocess;andtoobtaineffectivelearningperformancewithlittledatawhenbigdatasourcesmaynotbeavailable.
EnhancingtheperceptualcapabilitiesofAIsystemsPerceptionisanintelligentsystem'swindowintotheworld.
Perceptionbeginswith(possiblydistributed)sensordata,whichcomesindiversemodalitiesandforms,suchasthestatusofthesystemitselforinformationabouttheenvironment.
Sensordataareprocessedandfused,oftenalongwithaprioriknowledgeandmodels,toextractinformationrelevanttotheAIsystem'stasksuchas33NationalResearchCouncilComputerScienceTelecommunicationsBoard,ContinuingInnovationinInformationTechnology(TheNationalAcademiesPress,Washington,D.
C.
,2012),11;https://doi.
org/10.
17226/13427.
34https://www.
nitrd.
gov/PUBS/bigdatardstrategicplan.
pdfTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–10–geometricfeatures,attributes,location,andvelocity.
IntegrateddatafromperceptionformssituationalawarenesstoprovideAIsystemswiththecomprehensiveknowledgeandamodelofthestateoftheworldnecessarytoplanandexecutetaskseffectivelyandsafely.
AIsystemswouldgreatlybenefitfromadvancementsinhardwareandalgorithmstoenablemorerobustandreliableperception.
Sensorsmustbeabletocapturedataatlongerdistances,withhigherresolution,andinrealtime.
Perceptionsystemsneedtobeabletointegratedatafromavarietyofsensorsandothersources,includingthecomputationalcloud,todeterminewhattheAIsystemiscurrentlyperceivingandtoallowthepredictionoffuturestates.
Detection,classification,identification,andrecognitionofobjectsremainchallenging,especiallyunderclutteredanddynamicconditions.
Inaddition,perceptionofhumansmustbegreatlyimprovedbyusinganappropriatecombinationofsensorsandalgorithms,sothatAIsystemscanworkmoreeffectivelywithpeople.
10AframeworkforcalculatingandpropagatinguncertaintythroughouttheperceptionprocessisneededtoquantifytheconfidencelevelthattheAIsystemhasinitssituationalawarenessandtoimproveaccuracy.
UnderstandingtheoreticalcapabilitiesandlimitationsofAIWhiletheultimategoalformanyAIalgorithmsistoaddressopenchallengeswithhuman-likesolutions,wedonothaveagoodunderstandingofwhatthetheoreticalcapabilitiesandlimitationsareforAIandtheextenttowhichsuchhuman-likesolutionsareevenpossiblewithAIalgorithms.
TheoreticalworkisneededtobetterunderstandwhyAItechniques—especiallymachinelearning—oftenworkwellinpractice.
Whiledifferentdisciplines(includingmathematics,controlsciences,andcomputerscience)arestudyingthisissue,thefieldcurrentlylacksunifiedtheoreticalmodelsorframeworkstounderstandAIsystemperformance.
Additionalresearchisneededoncomputationalsolvability,whichisanunderstandingoftheclassesofproblemsthatAIalgorithmsaretheoreticallycapableofsolving,andlikewise,thosethattheyarenotcapableofsolving.
Thisunderstandingmustbedevelopedinthecontextofexistinghardware,inordertoseehowthehardwareaffectstheperformanceofthesealgorithms.
Understandingwhichproblemsaretheoreticallyunsolvablecanleadresearcherstodevelopapproximatesolutionstotheseproblems,orevenopenupnewlinesofresearchonnewhardwareforAIsystems.
Forexample,wheninventedinthe1960s,ArtificialNeuralNetworks(ANNs)couldonlybeusedtosolveverysimpleproblems.
ItonlybecamefeasibletouseANNstosolvecomplexproblemsafterhardwareimprovementssuchasparallelizationweremade,andalgorithmswereadjustedtomakeuseofthenewhardware.
Suchdevelopmentswerekeyfactorsinenablingtoday'ssignificantadvancesindeeplearning.
Pursuingresearchongeneral-purposeartificialintelligenceAIapproachescanbedividedinto"narrowAI"and"generalAI.
"NarrowAIsystemsperformindividualtasksinspecialized,well-defineddomains,suchasspeechrecognition,imagerecognition,andtranslation.
Severalrecent,highly-visible,narrowAIsystems,includingIBMWatsonandDeepMind'sAlphaGo,haveachievedmajorfeats.
35,36Indeed,theseparticularsystemshavebeenlabeled"superhuman"becausetheyhaveoutperformedthebesthumanplayersinJeopardy!
andGo,respectively.
ButthesesystemsexemplifynarrowAI,sincetheycanonlybeappliedtothetasksforwhichtheyarespecificallydesigned.
Usingthesesystemsonawiderrangeofproblemsrequiresasignificantre-engineeringeffort.
Incontrast,thelong-termgoalofgeneralAIistocreatesystemsthat35In2011,IBMWatsondefeatedtwoplayersconsideredamongthebesthumanplayersintheJeopardy!
game.
36In2016,AlphaGodefeatedthereigningworldchampionofGo,LeeSe-dol.
Notably,AlphaGocombinesdeeplearningandMonteCarlosearch—amethoddevelopedinthe1980s—whichitselfbuildsonaprobabilisticmethoddiscoveredinthe1940s.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–11–exhibittheflexibilityandversatilityofhumanintelligenceinabroadrangeofcognitivedomains,includinglearning,language,perception,reasoning,creativity,andplanning.
BroadlearningcapabilitieswouldprovidegeneralAIsystemstheabilitytotransferknowledgefromonedomaintoanotherandtointeractivelylearnfromexperienceandfromhumans.
GeneralAIhasbeenanambitionofresearcherssincetheadventofAI,butcurrentsystemsarestillfarfromachievingthisgoal.
TherelationshipbetweennarrowandgeneralAIiscurrentlybeingexplored;itispossiblethatlessonsfromonecanbeappliedtoimprovetheotherandviceversa.
Whilethereisnogeneralconsensus,mostAIresearchersbelievethatgeneralAIisstilldecadesaway,requiringalong-term,sustainedresearchefforttoachieveit.
DevelopingscalableAIsystemsGroupsandnetworksofAIsystemsmaybecoordinatedorautonomouslycollaboratetoperformtasksnotpossiblewithasingleAIsystem,andmayalsoincludehumansworkingalongsideorleadingtheteam.
Thedevelopmentanduseofsuchmulti-AIsystemscreatessignificantresearchchallengesinplanning,coordination,control,andscalabilityofsuchsystems.
Planningtechniquesformulti-AIsystemsmustbefastenoughtooperateandadaptinrealtimetochangesintheenvironment.
Theyshouldadaptinafluidmannertochangesinavailablecommunicationsbandwidthorsystemdegradationandfaults.
Manyprioreffortshavefocusedoncentralizedplanningandcoordinationtechniques;however,theseapproachesaresubjecttosinglepointsoffailure,suchasthelossoftheplanner,orlossofthecommunicationslinktotheplanner.
Distributedplanningandcontroltechniquesarehardertoachievealgorithmically,andareoftenlessefficientandincomplete,butpotentiallyoffergreaterrobustnesstosinglepointsoffailure.
Futureresearchmustdiscovermoreefficient,robust,andscalabletechniquesforplanning,control,andcollaborationofteamsofmultipleAIsystemsandhumans.
Fosteringresearchonhuman-likeAIAttaininghuman-likeAIrequiressystemstoexplainthemselvesinwaysthatpeoplecanunderstand.
Thiswillresultinanewgenerationofintelligentsystems,suchasintelligenttutoringsystemsandintelligentassistantsthatareeffectiveinassistingpeoplewhenperformingtheirtasks.
Thereisasignificantgap,however,betweenthewaycurrentAIalgorithmsworkandhowpeoplelearnandperformtasks.
Peoplearecapableoflearningfromjustafewexamples,orbyreceivingformalinstructionand/or"hints"toperformingtasks,orbyobservingotherpeopleperformingthosetasks.
Medicalschoolstakethisapproach,forexample,whenmedicalstudentslearnbyobservinganestablisheddoctorperformingacomplexmedicalprocedure.
Eveninhigh-performancetaskssuchasworld-championshipGogames,amaster-levelplayerwouldhaveplayedonlyafewthousandgamestotrainhim/herself.
Incontrast,itwouldtakehundredsofyearsforahumantoplaythenumberofgamesneededtotrainAlphaGo.
Morefoundationalresearchonnewapproachesforachievinghuman-likeAIwouldbringthesesystemsclosertothisgoal.
DevelopingmorecapableandreliablerobotsSignificantadvancesinrobotictechnologiesoverthelastdecadeareleadingtopotentialimpactsinamultiplicityofapplications,includingmanufacturing,logistics,medicine,healthcare,defenseandnationalsecurity,agriculture,andconsumerproducts.
Whilerobotswerehistoricallyenvisionedforstaticindustrialenvironments,recentadvancesinvolveclosecollaborationsbetweenrobotsandhumans.
Roboticstechnologiesarenowshowingpromiseintheirabilitytocomplement,augment,enhance,oremulatehumanphysicalcapabilitiesorhumanintelligence.
However,scientistsneedtomaketheseroboticsystemsmorecapable,reliable,andeasy-to-use.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–12–Researchersneedtobetterunderstandroboticperceptiontoextractinformationfromavarietyofsensorstoproviderobotswithreal-timesituationalawareness.
Progressisneededincognitionandreasoningtoallowrobotstobetterunderstandandinteractwiththephysicalworld.
Animprovedabilitytoadaptandlearnwillallowrobotstogeneralizetheirskills,performself-assessmentoftheircurrentperformance,andlearnarepertoireofphysicalmovementsfromhumanteachers.
Mobilityandmanipulationareareasforfurtherinvestigationsothatrobotscanmoveacrossruggedanduncertainterrainandhandleavarietyofobjectsdexterously.
Robotsneedtolearntoteamtogetherinaseamlessfashionandcollaboratewithhumansinawaythatistrustworthyandpredictable.
AdvancinghardwareforimprovedAIWhileAIresearchismostcommonlyassociatedwithadvancesinsoftware,theperformanceofAIsystemshasbeenheavilydependentonthehardwareuponwhichitruns.
ThecurrentrenaissanceindeepmachinelearningisdirectlytiedtoprogressinGPU-basedhardwaretechnologyanditsimprovedmemory,37input/output,clockspeeds,parallelism,andenergyefficiency.
DevelopinghardwareoptimizedforAIalgorithmswillenableevenhigherlevelsofperformancethanGPUs.
Oneexampleis"neuromorphic"processorsthatarelooselyinspiredbytheorganizationofthebrainand,insomecases,optimizedfortheoperationofneuralnetworks.
38HardwareadvancescanalsoimprovetheperformanceofAImethodsthatarehighlydata-intensive.
Furtherstudyofmethodstoturnonandoffdatapipelinesincontrolledwaysthroughoutadistributedsystemiscalledfor.
Continuedresearchisalsoneededtoallowmachinelearningalgorithmstoefficientlylearnfromhigh-velocitydata,includingdistributedmachinelearningalgorithmsthatsimultaneouslylearnfrommultipledatapipelines.
Moreadvancedmachinelearning-basedfeedbackmethodswillallowAIsystemstointelligentlysampleorprioritizedatafromlarge-scalesimulations,experimentalinstruments,anddistributedsensorsystems,suchasSmartBuildingsandtheInternetofThings(IoT).
SuchmethodsmayrequiredynamicI/Odecision-making,inwhichchoicesaremadeinrealtimetostoredatabasedonimportanceorsignificance,ratherthansimplystoringdataatfixedfrequencies.
CreatingAIforimprovedhardwareWhileimprovedhardwarecanleadtomorecapableAIsystems,AIsystemscanalsoimprovetheperformanceofhardware.
39Thisreciprocitywillleadtofurtheradvancesinhardwareperformance,sincephysicallimitsoncomputingrequirenovelapproachestohardwaredesigns.
40AI-basedmethodscouldbeespeciallyimportantforimprovingtheoperationofhigh-performancecomputing(HPC)systems.
Suchsystemsconsumevastquantitiesofenergy.
AIisbeingusedtopredictHPCperformanceandresourceusage,andtomakeonlineoptimizationdecisionsthatincreaseefficiency;moreadvancedAItechniquescouldfurtherenhancesystemperformance.
AIcanalsobeusedtocreate37GPUstandsforgraphicsprocessingunit,whichisapower-andcost-efficientprocessorincorporatinghundredsofprocessingcores;thisdesignmakesitespeciallywellsuitedforinherentlyparallelapplications,includingmostAIsystems.
38Neuromorphiccomputingreferstotheabilityofhardwaretolearn,adapt,andphysicallyreconfigure,takinginspirationfrombiologyorneuroscience.
39M.
MilanoandL.
Benini,"PredictiveModelingforJobPowerConsumptioninHPCSystems,"InProceedingsofHighPerformanceComputing:31stInternationalConference,ISCHighPerformance2016(SpringerVol.
9697,2016).
40ThesephysicallimitsoncomputingarecalledDennardscaling,andleadtohighon-chippowerdensitiesandthephenomenoncalled"darksilicon",wheredifferentpartsofachipneedtobeturnedoffinordertolimittemperaturesandensuredataintegrity.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–13–self-reconfigurableHPCsystemsthatcanhandlesystemfaultswhentheyoccur,withouthumanintervention.
41ImprovedAIalgorithmscanincreasetheperformanceofmulti-coresystemsbyreducingdatamovementsbetweenprocessorsandmemory—theprimaryimpedimenttoexascalecomputingsystemsthatoperate10timesfasterthantoday'ssupercomputers.
42Inpractice,theconfigurationofexecutionsinHPCsystemsareneverthesame,anddifferentapplicationsareexecutedconcurrently,withthestateofeachdifferentsoftwarecodeevolvingindependentlyintime.
AIalgorithmsneedtobedesignedtooperateonlineandatscaleforHPCsystems.
41A.
Cocaa-Fernández,J.
Ranilla,andL.
Sánchez,"Energy-efficientallocationofcomputingnodeslotsinHPCclustersthroughparameterlearningandhybridgeneticfuzzysystemmodeling,"JournalofSupercomputing71(2015):1163-1174.
42Exascalecomputingsystemscanachieveatleastabillionbillioncalculationspersecond.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–14–Strategy2:DevelopEffectiveMethodsforHuman-AICollaboration2019UpdateDevelopingAIsystemsthatcomplementandaugmenthumancapabilities,withincreasingfocusonthefutureofworkSincethereleaseofthe2016NationalAIR&DStrategicPlan,nationalinteresthasgrowninhuman-AIcollaboration.
WhenAIsystemscomplementandaugmenthumancapabilities,humansandAIbecomepartnersacrossarangeofsharedtofullyautonomousscenarios.
Inparticular,human-AIcollaborationhasbeenelevatedasbothachallengeandanopportunityinthecontextofthefutureofwork.
Inthepastthreeyears,newlyestablishedaswellaslongstandingconferences,workshops,andtaskforceshaveprioritizedhuman-AIcollaborationbroadly.
Forexample,theConferenceonHumanComputationandCrowdsourcinghasgrownfromaworkshoptoamajorinternationalconferencethatfostersresearchintheintersectionofAIandhuman-computerinteraction(HCI).
45In2018,theAssociationfortheAdvancementofArtificialIntelligenceselectedhuman-AIcollaborationastheemergingtopicforitsannualconference.
46InMay2019,thelargestconferenceonhuman-computerinteraction,CHI,includedaworkshopon"BridgingtheGapBetweenAIandHCI.
"47ThejournalHuman-ComputerInteractionputoutacallinMarch2019forsubmissionsforaspecialissueon"unifyinghuman-computerinteractionandartificialintelligence.
"48Human-AICollaboration:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,severalagencieshaveinitiatedeffortsforStrategy2:NSF'sFutureofWorkattheHuman-TechnologyFrontier43BigIdeaissupportingsocio-technicalresearchenablingafuturewhereintelligenttechnologiescollaboratesynergisticallywithhumanstoachievebroadparticipationintheworkforceandimprovethesocial,economic,andenvironmentalbenefitsacrossarangeofworksettings.
NOAA(NationalOceanographicandAtmosphericAdministration)isadvancinghuman-AIcollaborationforhurricane,tornado,andothersevereweatherpredictionswherethehumanforecasterandanAIsystemworktogethertoimprovesevereweatherwarninggenerationandtoidentifydistinctpatternsthatareprecursorstoextremeevents.
Sometimesreferredtoas"humansabovetheloop,"humanforecastersoverseetheAIsystem'spredictionsanddirecttheoutcomes.
NIHhasongoingresearchinnaturallanguageprocessingbasedonadatabaseof96.
3millionfactsextractedfromallMEDLINEcitationsmaintainedbytheNationalLibraryofMedicine.
A2019DOEworkshopreportonScientificMachineLearningidentifiedpriorityresearchdirections,majorscientificusecases,andtheemergingtrendthathuman-AIcollaborationswilltransformthewayscienceisdone.
4443https://www.
nsf.
gov/eng/futureofwork.
jsp44DOEworkshopreport,BasicResearchNeedsforScientificMachineLearning:CoreTechnologiesforArtificialIntelligence:https://www.
osti.
gov/biblio/1478744.
45WelcometoHCOMP2019:https://www.
humancomputation.
com/.
46AAAI-18EmergingTopicHuman-AICollaboration:http://www.
aaai.
org/Conferences/AAAI/2018/aaai18emergingcall.
php.
47WhereistheHumanBridgingtheGapBetweenAIandHCI:CHI2019Workshop:https://michae.
lv/ai-hci-workshop/.
48Call:"UnifyingHumanComputerInteractionandArtificialIntelligence"issueofHuman-ComputerInteraction:https://ispr.
info/2019/02/20/call-unifying-human-computer-interaction-and-artificial-intelligence-issue-of-human-computer-interaction/.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–15–Inthecontextofwork,conferenceshaveemergedexploringtheroleofthehuman,themachine,andtheirpartnership,suchasMIT'sComputerScienceandArtificialIntelligenceLab(CSAIL)andtheInitiativeontheDigitalEconomythatlaunchedtheAnnualAIandtheFutureofWorkCongress.
49,50AspartofA20-YearCommunityRoadmapforArtificialIntelligenceResearchintheU.
S.
,51in2019theComputingCommunityConsortium(CCC)heldaworkshopfocusedonmeaningfulinteractionbetweenhumansandAIsystems.
52Additionally,theCCCoperatedtheHumanTechnologyFrontiertaskforcein2017-2018tofocusonthepotentialoftechnologytoaugmenthumanperformancein,includingbutnotlimitedto,theworkplace,theclassroom,andthehealthcaresystem.
53Thecross-strategyprincipleinthe2016NationalAIR&DStrategicPlan,"appropriatetrustofAIsystemsrequiresexplainability,especiallyastheAIgrowsinscaleandcomplexity,"hasseenanR&Dcalltoactioninthecontextofhuman-AIcollaborations.
Thisprinciplehasbeenidentifiedbyanumberofprofessionalsocietiesandagenciesasapriorityarea(seesidebar).
ThisresearchareareflectstheintersectionofStrategies2and3,asexplainability,fairness,andtransparencyarekeyprinciplesforAIsystemstoeffectivelycollaboratewithhumans.
Likewise,thechallengeofunderstandinganddesigninghuman-AIethicsandvaluealignmentintosystemsremainsanopenresearcharea.
Inparallel,theprivatesectorhasrespondedwithprinciplesforeffectivehuman-AIcollaboration.
54,55AsFederalagencieshaveincreasedAIinvestmentsinthepastthreeyearsalongmissionobjectives,theyhavesharedacommonemphasisonhuman-machinecognition,autonomy,andagency,suchasindecisionsupport,riskmodeling,situationalawareness,andtrustedmachineintelligence(seesidebar).
ThroughsuchR&Dinvestments,researchpartnershipsaregrowingacrossanumberofaxes,bringingtogethercomputationalscientists;behavioral,cognitive,andpsychologicalscientists;andscientistsandengineersfromotherdomains.
NewcollaborationshaveformedbetweenacademicresearchersandusersofAIsystemsinsideandoutsidetheworkplace.
Movingforward,itiscriticalthatFederalagenciescontinuetofosterAIR&DintheopenworldtopromotethedesignofAIsystemsthatincorporateandaccommodatethesituationsandgoalsofuserssothatAIsystemsanduserscanworkcollaborativelyinbothanticipatedandunanticipatedcircumstances.
WhilecompletelyautonomousAIsystemswillbeimportantinsomeapplicationdomains(e.
g.
,underwaterordeepspaceexploration),manyotherapplicationareas(e.
g.
,disasterrecoveryandmedicaldiagnostics)aremosteffectivelyaddressedbyacombinationofhumansandAIsystemsworkingtogethertoachieveapplicationgoals.
ThiscollaborativeinteractiontakesadvantageofthecomplementarynatureofhumansandAIsystems.
Whileeffectiveapproachesforhuman-AIcollaborationalreadyexist,mostoftheseare"pointsolutions"thatonlyworkinspecificenvironmentsusingspecificplatformstowardspecificgoals.
Generatingpointsolutionsforeverypossibleapplicationinstancedoesnotscale;moreworkisthusneededtogobeyondthesepointsolutionstowardmore49https://futureofwork.
csail.
mit.
edu/.
50AIandFutureofWorkInnovationSummit2019:https://analyticsevent.
com/.
51https://cra.
org/ccc/wp-content/uploads/sites/2/2019/03/AI_Roadmap_Exec_Summary-FINAL-.
pdf52ArtificialIntelligenceRoadmapWorkshop2–Interaction:https://cra.
org/ccc/events/artificial-intelligence-roadmap-workshop-2-interaction/.
53https://cra.
org/ccc/human-technology-frontier/54https://www.
microsoft.
com/en-us/research/uploads/prod/2019/01/Guidelines-for-Human-AI-Interaction-camera-ready.
pdf55https://www.
partnershiponai.
org/about/#our-workTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–16–generalmethodsofhuman-AIcollaboration.
Thetradeoffsmustbeexploredbetweendesigninggeneralsystemsthatworkinalltypesofproblems,requiringlesshumanefforttobuildandgreaterfacilityforswitchingbetweenapplications,versusbuildingalargenumberofproblem-specificsystemsthatmayworkmoreeffectivelyforeachproblem.
FutureapplicationswillvaryconsiderablyinthefunctionalroledivisionsbetweenhumansandAIsystems,thenatureoftheinteractionsbetweenhumansandAIsystems,thenumberofhumansandotherAIsystemsworkingtogether,andhowhumansandAIsystemswillcommunicateandsharesituationalawareness.
FunctionalroledivisionsbetweenhumansandAIsystemstypicallyfallintooneofthefollowingcategories:1.
AIperformsfunctionsalongsidethehuman:AIsystemsperformperipheraltasksthatsupportthehumandecisionmaker.
Forexample,AIcanassisthumanswithworkingmemory,shortorlong-termmemoryretrieval,andpredictiontasks.
2.
AIperformsfunctionswhenthehumanencountershighcognitiveoverload:AIsystemsperformcomplexmonitoringfunctions(suchasgroundproximitywarningsystemsinaircraft),decisionmaking,andautomatedmedicaldiagnoseswhenhumansneedassistance.
3.
AIperformsfunctionsinlieuofahuman:AIsystemsperformtasksforwhichhumanshaveverylimitedcapabilities,suchasforcomplexmathematicaloperations,controlguidancefordynamicsystemsincontestedoperationalenvironments,aspectsofcontrolforautomatedsystemsinharmfulortoxicenvironments,andinsituationswhereasystemshouldrespondveryrapidly(e.
g.
,innuclearreactorcontrolrooms).
AchievingeffectiveinteractionsbetweenhumansandAIsystemsrequiresadditionalR&Dtoensurethatthesystemdesigndoesnotleadtoexcessivecomplexity,undertrust,orovertrust.
ThefamiliarityofhumanswithAIsystemscanbeincreasedthroughtrainingandexperience,toensurethatthehumanhasagoodunderstandingoftheAIsystem'scapabilitiesandwhattheAIsystemcanandcannotdo.
Toaddresstheseconcerns,certainhuman-centeredautomationprinciplesshouldbeusedinthedesignanddevelopmentofthesesystems:561.
Employintuitive,user-friendlydesignofhuman-AIsysteminterfaces,controls,anddisplays.
2.
Keeptheoperatorinformed.
Displaycriticalinformation,statesoftheAIsystem,andchangestothesestates.
3.
Keeptheoperatortrained.
Engageinrecurrenttrainingforgeneralknowledge,skills,andabilities(KSAs),aswellastraininginalgorithmsandlogicemployedbyAIsystemsandtheexpectedfailuremodesofthesystem.
4.
Makeautomationflexible.
DeployingAIsystemsshouldbeconsideredasadesignoptionforoperatorswhowishtodecidewhethertheywanttousethemornot.
AlsoimportantisthedesignanddeploymentofadaptiveAIsystemsthatcanbeusedtosupporthumanoperatorsduringperiodsofexcessiveworkloadorfatigue.
57,58Manyfundamentalchallengesariseforresearcherswhencreatingsystemsthatworkeffectivelywithhumans.
Severaloftheseimportantchallengesareoutlinedinthefollowingsubsections.
56C.
WickensandJ.
G.
Hollands,"Attention,time-sharing,andworkload.
"InEngineering,PsychologyandHumanPerformance(London:PearsonPLC,1999),439-479.
57https://www.
nasa.
gov/mission_pages/SOFIA/index.
html58https://cloud1.
arc.
nasa.
gov/intex-na/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–17–Seekingnewalgorithmsforhuman-awareAIOvertheyears,AIalgorithmshavebecomeabletosolveproblemsofincreasingcomplexity.
However,thereisagapbetweenthecapabilitiesofthesealgorithmsandtheusabilityofthesesystemsbyhumans.
Human-awareintelligentsystemsareneededthatcaninteractintuitivelywithusersandenableseamlessmachine-humancollaborations.
Intuitiveinteractionsincludeshallowinteractions,suchaswhenauserdiscardsanoptionrecommendedbythesystem;model-basedapproachesthattakeintoaccounttheusers'pastactions;orevendeepmodelsofuserintentthatarebaseduponaccuratehumancognitivemodels.
Interruptionmodelsmustbedevelopedthatallowanintelligentsystemtointerruptthehumanonlywhennecessaryandappropriate.
Intelligentsystemsshouldalsohavetheabilitytoaugmenthumancognition,knowingwhichinformationtoretrievewhentheuserneedsit,evenwhentheyhavenotpromptedthesystemexplicitlyforthatinformation.
Futureintelligentsystemsmustbeabletoaccountforhumansocialnormsandactaccordingly.
Intelligentsystemscanmoreeffectivelyworkwithhumansiftheypossesssomedegreeofemotionalintelligence,sothattheycanrecognizetheirusers'emotionsandrespondappropriately.
Anadditionalresearchgoalistogobeyondinteractionsofonehumanandonemachine,towarda"systems-of-systems",thatis,teamscomposedofmultiplemachinesinteractingwithmultiplehumans.
Human-AIsysteminteractionshaveawiderangeofobjectives.
AIsystemsneedtheabilitytorepresentamultitudeofgoals,actionsthattheycantaketoreachthosegoals,constraintsonthoseactions,andotherfactors,aswellaseasilyadapttomodificationsinthegoals.
Inaddition,humansandAIsystemsmustsharecommongoalsandhaveamutualunderstandingofthemandrelevantaspectsoftheircurrentstates.
Furtherinvestigationisneededtogeneralizethesefacetsofhuman-AIsystemstodevelopsystemsthatrequirelesshumanengineering.
DevelopingAItechniquesforhumanaugmentationWhilemuchofthepriorfocusofAIresearchhasbeenonalgorithmsthatmatchoroutperformpeopleperformingnarrowtasks,moreworkisneededtodevelopsystemsthataugmenthumancapabilitiesacrossmanydomains.
Humanaugmentationresearchincludesalgorithmsthatworkonastationarydevice(suchasacomputer);wearabledevices(suchassmartglasses);implanteddevices(suchasbraininterfaces);andinspecificuserenvironments(suchasspeciallytailoredoperatingrooms).
Forexample,augmentedhumanawarenesscouldenableamedicalassistanttopointoutamistakeinamedicalprocedure,basedondatareadingscombinedfrommultipledevices.
Othersystemscouldaugmenthumancognitionbyhelpingtheuserrecallpastexperiencesapplicabletotheuser'scurrentsituation.
AnothertypeofcollaborationbetweenhumansandAIsystemsinvolvesactivelearningforintelligentdataunderstanding.
Inactivelearning,inputissoughtfromadomainexpertandlearningisonlyperformedondatawhenthelearningalgorithmisuncertain.
Thisisanimportanttechniquetoreducetheamountoftrainingdatathatneedstobegeneratedinthefirstplace,ortheamountthatneedstobelearned.
Activelearningisalsoakeywaytoobtaindomainexpertinputandincreasetrustinthelearningalgorithm.
Activelearninghassofaronlybeenusedwithinsupervisedlearning;furtherresearchisneededtoincorporateactivelearningintounsupervisedlearning(e.
g.
,clustering,anomalydetection)andreinforcementlearning.
59Probabilisticnetworksallowdomainknowledgetobeincludedintheformofpriorprobabilitydistributions.
Generalwaysofallowingmachinelearningalgorithmstoincorporatedomainknowledgemustbesought,whetherintheformofmathematicalmodels,text,orothers.
59Whilesupervisedlearningrequireshumanstoprovidetheground-truthanswers,reinforcementlearningandunsupervisedlearningdonot.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–18–Developingtechniquesforvisualizationandhuman-AIinterfacesBettervisualizationanduserinterfacesareadditionalareasthatneedmuchgreaterdevelopmenttohelphumansunderstandlarge-volumemoderndatasetsandinformationcomingfromavarietyofsources.
Visualizationanduserinterfacesmustclearlypresentincreasinglycomplexdataandinformationderivedfromtheminahuman-understandableway.
Providingreal-timeresultsisimportantinsafety-criticaloperationsandmaybeachievedwithincreasingcomputationalpowerandconnectedsystems.
Inthesetypesofsituations,usersneedvisualizationanduserinterfacesthatcanquicklyconveythecorrectinformationforreal-timeresponse.
Human-AIcollaborationcanbeappliedinawidevarietyofenvironments,andwherethereareconstraintsoncommunication.
Insomedomains,human-AIcommunicationlatenciesarelowandcommunicationisrapidandreliable.
Inotherdomains(e.
g.
,NASA'sdeploymentoftheroversSpiritandOpportunitytoMars),remotecommunicationbetweenhumansandtheAIsystemhasaveryhighlatency(e.
g.
,roundtriptimesof5-20minutesbetweenEarthandMars),thusrequiringthedeployedplatform(s)tooperatelargelyautonomously,withonlyhigh-levelstrategicgoalscommunicatedtotheplatform.
ThesecommunicationsrequirementsandconstraintsareimportantconsiderationsfortheR&Dofuserinterfaces.
DevelopingmoreeffectivelanguageprocessingsystemsEnablingpeopletointeractwithAIsystemsthroughspokenandwrittenlanguagehaslongbeenagoalofAIresearchers.
Whilesignificantadvanceshavebeenmade,considerableopenresearchchallengesmustbeaddressedinlanguageprocessingbeforehumanscancommunicateaseffectivelywithAIsystemsastheydowithotherhumans.
Muchrecentprogressinlanguageprocessinghasbeencreditedtotheuseofdata-drivenmachinelearningapproaches,whichhaveresultedinsuccessfulsystemsthat,forexample,successfullyrecognizefluentEnglishspeechinquietsurroundingsinrealtime.
Theseachievements,however,areonlyfirststepstowardreachinglonger-termgoals.
Currentsystemscannotdealwithreal-worldchallengessuchasspeechinnoisysurroundings,heavilyaccentedspeech,children'sspeech,impairedspeech,andspeechforsignlanguages.
Thedevelopmentoflanguageprocessingsystemscapableofengaginginreal-timedialoguewithhumansisalsoneeded.
Suchsystemswillneedtoinferthegoalsandintentionsofitshumaninterlocutors,usetheappropriateregister,styleandrhetoricforthesituation,andemployrepairstrategiesincaseofdialoguemisunderstandings.
Furtherresearchisneededondevelopingsystemsthatmoreeasilygeneralizeacrossdifferentlanguages.
Additionally,morestudyisrequiredonacquiringusefulstructureddomainknowledgeinaformreadilyaccessiblebylanguageprocessingsystems.
LanguageprocessingadvancesinmanyotherareasarealsoneededtomakeinteractionsbetweenhumansandAIsystemsmorenaturalandintuitive.
Robustcomputationalmodelsmustbebuiltforpatternsinbothspokenandwrittenlanguagethatprovideevidenceforemotionalstate,affect,andstance,andfordeterminingtheinformationthatisimplicitinspeechandtext.
NewlanguageprocessingtechniquesareneededforgroundinglanguageintheenvironmentalcontextforAIsystemsthatoperateinthephysicalworld,suchasinrobotics.
Finally,sincethemannerinwhichpeoplecommunicateinonlineinteractionscanbequitedifferentfromvoiceinteractions,modelsoflanguagesusedinthesecontextsmustbeperfectedsothatsocialAIsystemscaninteractmoreeffectivelywithpeople.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–19–Strategy3:UnderstandandAddresstheEthical,Legal,andSocietalImplicationsofAI2019UpdateAddressingethical,legal,andsocietalconsiderationsinAISincethereleaseofthe2016NationalAIR&DStrategicPlan,R&Dactivitiesaddressingtheethical,legal,andsocietalimplicationsofAIsystemdevelopmentanddeploymenthaveincreased.
ThereisagrowingrealizationthatAIsystemsmustbe"trustworthy,"andthatAIcouldtransformmanysectorsofsocialandeconomiclife,includingemployment,healthcare,andmanufacturing.
InternationalorganizationssuchastheOrganisationforEconomicCo-operationandDevelopment(OECD)63andtheG7InnovationMinisters64haveencouragedR&DtoincreasetrustinandadoptionofAI.
The2016NationalAIR&DStrategicPlanwasprescientinidentifyingresearchthemesinprivacy;improvingfairness,transparency,andaccountabilityofAIsystemsbydesign;anddesigningarchitecturesforethicalAI.
Researchconferencesdedicatedtofairness,accountability,andtransparencyinMLandAIsystemshaveflourished.
65Federalagencieshaverespondedwithavarietyofnewresearchprogramsandmeetingsfocusedonthesecriticalareas(seesidebar).
Explainability,fairness,andtransparency:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,anumberofagencieshaveinitiatedAIR&DprogramsforStrategy3:DARPA'sExplainableAI(XAI)program60aimstocreateasuiteofMLtechniquesthatproducemoreexplainableAIsystemswhilemaintainingahighleveloflearningperformance(predictionaccuracy).
XAIwillalsoenablehumanuserstounderstand,appropriatelytrust,andeffectivelymanagetheemerginggenerationofAIsystems.
Moregenerally,DoDiscommittedto"leadinginmilitaryethicsandAIsafety"asoneoffivekeyactionsoutlinedinthestrategicapproachthatguidesitseffortstoacceleratetheadoptionofAIsystems.
61NSFandAmazonarecollaborating62tojointlysupportresearchfocusedonAIfairnesswiththegoalofcontributingtotrustworthyAIsystemsthatarereadilyacceptedanddeployedtotacklegrandchallengesfacingsociety.
Specifictopicsofinterestinclude,butarenotlimitedto,transparency,explainability,accountability,potentialadversebiasesandeffects,mitigationstrategies,validationoffairness,andconsiderationsofinclusivity.
60https://www.
darpa.
mil/program/explainable-artificial-intelligence61"Summaryofthe2018DepartmentofDefenseArtificialIntelligenceStrategy":https://media.
defense.
gov/2019/Feb/12/2002088963/-1/-1/1/SUMMARY-OF-DOD-AI-STRATEGY.
PDF.
62https://www.
nsf.
gov/pubs/2019/nsf19571/nsf19571.
htm63"OECDInitiativesonAI":http://www.
oecd.
org/going-digital/ai/oecd-initiatives-on-ai.
htm.
64"G7InnovationMinisters'StatementonAI":http://www.
g8.
utoronto.
ca/employment/2018-labour-annex-b-en.
html.
65http://www.
fatml.
org/;https://fatconference.
org/;http://www.
aies-conference.
com/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–20–The2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligenceemphasizesthatmaintainingAmericanleadershipinAIrequiresaconcertedefforttopromoteadvancementsintechnologyandinnovation,whileprotectingcivilliberties,privacy,andAmericanvalues:1TheUnitedStatesmustfosterpublictrustandconfidenceinAItechnologiesandprotectcivilliberties,privacy,andAmericanvaluesintheirapplicationinordertofullyrealizethepotentialofAItechnologiesfortheAmericanpeople.
MoreR&DisneededtodevelopAIarchitecturesthatincorporateethical,legal,andsocietalconcernsthroughtechnicalmechanismssuchastransparencyandexplainability.
ThisR&Dwillrequireintensivecollaborationamongtechnicalexpertsaswellasstakeholdersandspecialistsinotherfieldsincludingthesocialandbehavioralsciences,law,ethics,andphilosophy.
Sinceethicaldecisionsmayalsobeheavilycontext-orapplication-dependent,collaborationwithdomainexpertscouldberequiredaswell.
Thisinterdisciplinaryapproachcouldbeincorporatedinthetraining,design,testing,evaluation,andimplementationofAIintheinterestsofunderstandingandaccountingforAI-induceddecisionsandactionsandmitigatingunintendedconsequences.
FederalagenciesshouldthereforecontinuetofosterthegrowingcommunityofinterestinfurtherR&Doftheseissuesbysponsoringresearchandconveningexpertsandstakeholders.
WhenAIagentsactautonomously,weexpectthemtobehaveaccordingtotheformalandinformalnormstowhichweholdourfellowhumans.
Asfundamentalsocialorderingforces,lawandethicsthereforebothinformandadjudgethebehaviorofAIsystems.
Thedominantresearchneedsinvolvebothunderstandingtheethical,legal,andsocialimplicationsofAI,aswellasdevelopingmethodsforAIdesignthatalignwithethical,legal,andsocialprinciples.
Privacyconcernsmustalsobetakenintoaccount;furtherinformationonthisissuecanbefoundintheNationalPrivacyResearchStrategy.
66Aswithanytechnology,theacceptableusesofAIwillbeinformedbythetenetsoflawandethics;thechallengeishowtoapplythosetenetstothisnewtechnology,particularlythoseinvolvingautonomy,agency,andcontrol.
Asilluminatedin"ResearchPrioritiesforRobustandBeneficialArtificialIntelligence,"67Inordertobuildsystemsthatrobustlybehavewell,weofcourseneedtodecidewhatgoodbehaviormeansineachapplicationdomain.
Thisethicaldimensionistiedintimatelytoquestionsofwhatengineeringtechniquesareavailable,howreliablethesetechniquesare,andwhattrade-offsaremade—allareaswherecomputerscience,machinelearning,andbroaderAIexpertiseisvaluable.
Researchinthisareacanbenefitfrommultidisciplinaryperspectivesthatinvolveexpertsfromcomputerscience,socialandbehavioralsciences,ethics,biomedicalscience,psychology,economics,law,andpolicyresearch.
FurtherinvestigationisneededinareasbothinsideandoutsideoftheNITRD-relevantITdomain(i.
e.
,ininformationtechnology,aswellasinthedisciplinesmentionedpreviously)toinformtheR&DanduseofAIsystemsandtheirimpactsonsociety.
Thefollowingsubsectionsexplorekeyinformationtechnologyresearchchallengesinthisarea.
66https://www.
nitrd.
gov/pubs/NationalPrivacyResearchStrategy.
pdf67"AnOpenLetter:ResearchPrioritiesforRobustandBeneficialArtificialIntelligence"(FutureofLifeInstitute):http://futureoflife.
org/ai-open-letter/.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–21–Improvingfairness,transparency,andaccountabilitybydesignManyconcernshavebeenvoicedaboutthesusceptibilityofdata-intensiveAIalgorithmstoerrorandmisuse,andthepossibleramificationsforgender,age,racial,oreconomicclasses.
ThepropercollectionanduseofdataforAIsystems,inthisregard,representanimportantchallenge.
Beyondpurelydata-relatedissues,however,largerquestionsariseaboutthedesignofAItobeinherentlyjust,fair,transparent,andaccountable.
Researchersmustlearnhowtodesignthesesystemssothattheiractionsanddecision-makingaretransparentandeasilyinterpretablebyhumans,andthuscanbeexaminedforanybiastheymaycontain,ratherthanjustlearningandrepeatingthesebiases.
Thereareseriousintellectualissuesabouthowtorepresentand"encode"valueandbeliefsystems.
Scientistsmustalsostudytowhatextentjusticeandfairnessconsiderationscanbedesignedintothesystem,andhowtoaccomplishthiswithintheboundsofcurrentengineeringtechniques.
BuildingethicalAIBeyondfundamentalassumptionsofjusticeandfairnessareotherconcernsaboutwhetherAIsystemscanexhibitbehaviorthatabidesbygeneralethicalprinciples.
HowmightadvancesinAIframenew"machine-relevant"questionsinethics,orwhatusesofAImightbeconsideredunethicalEthicsisinherentlyaphilosophicalquestionwhileAItechnologydependson,andislimitedby,engineering.
Withinthelimitsofwhatistechnologicallyfeasible,therefore,researchersmuststrivetodevelopalgorithmsandarchitecturesthatareverifiablyconsistentwith,orconformto,existinglaws,socialnormsandethics—clearlyaverychallengingtask.
Ethicalprinciplesaretypicallystatedwithvaryingdegreesofvaguenessandarehardtotranslateintoprecisesystemandalgorithmdesign.
TherearealsocomplicationswhenAIsystems,particularlywithnewkindsofautonomousdecision-makingalgorithms,facemoraldilemmasbasedonindependentandpossiblyconflictingvaluesystems.
Ethicalissuesvaryaccordingtoculture,religion,andbeliefs.
However,acceptableethicsreferenceframeworkscanbedevelopedtoguideAIsystemreasoninganddecision-makinginordertoexplainandjustifyitsconclusionsandactions.
Amultidisciplinaryapproachisneededtogeneratedatasetsfortrainingthatreflectanappropriatevaluesystem,includingexamplesthatindicatepreferredbehaviorwhenpresentedwithdifficultmoralissuesorwithconflictingvalues.
Theseexamplescanincludelegalorethical"cornercases,"labeledbyanoutcomeorjudgmentthatistransparenttotheuser.
68AIneedsadequatemethodsforvalues-basedconflictresolution,wherethesystemincorporatesprinciplesthatcanaddresstherealitiesofcomplexsituationswherestrictrulesareimpracticable.
DesigningarchitecturesforethicalAIAdditionalprogressinfundamentalresearchmustbemadetodeterminehowtobestdesignarchitecturesforAIsystemsthatincorporateethicalreasoning.
Avarietyofapproacheshavebeensuggested,suchasatwo-tiermonitorarchitecturethatseparatestheoperationalAIfromamonitoragentthatisresponsiblefortheethicalorlegalassessmentofanyoperationalaction.
68Analternativeviewisthatsafetyengineeringispreferred,inwhichapreciseconceptualframeworkfortheAIagentarchitectureisusedtoensurethatAIbehaviorissafeandnotharmfultohumans.
69Athirdmethodistoformulateanethicalarchitectureusingsettheoreticprinciples,combinedwithlogicalconstraints68A.
EtzioniandO.
Etzioni,"DesigningAISystemsthatObeyOurLawsandValues,"CommunicationsoftheACM59(9)(2016):29-31.
69R.
Y.
Yampolsky,"ArtificialIntelligenceSafetyEngineering:WhyMachineEthicsisaWrongApproach.
"InPhilosophyandTheoryofArtificialIntelligence,ed.
V.
C.
Muller(Heidelberg:SpringerVerlag,2013),389-396.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–22–onAIsystembehaviorthatrestrictactiontoconformtoethicaldoctrine.
70AsAIsystemsbecomemoregeneral,theirarchitectureswilllikelyincludesubsystemsthatcantakeonethicalissuesatmultiplelevelsofjudgment,including:71rapidresponsepatternmatchingrules,deliberativereasoningforslowerresponsesfordescribingandjustifyingactions,socialsignalingtoindicatetrustworthinessfortheuser,andsocialprocessesthatoperateoverevenlongertimescalestoenablethesystemtoabidebyculturalnorms.
ResearcherswillneedtofocusonhowtobestaddresstheoveralldesignofAIsystemsthatalignwithethical,legal,andsocietalgoals.
70R.
C.
Arkin,"GoverningLegalBehavior:EmbeddingEthicsinaHybridDeliberative/ReactiveRobotArchitecture,"GeorgiaInstituteofTechnologyTechnicalReport,GIT-GVU-07-11,2007.
71B.
Kuipers,"Human-likeMoralityandEthicsforRobots,"AAAI-16WorkshoponAI,EthicsandSociety,2016;https://web.
eecs.
umich.
edu/~kuipers/papers/Kuipers-aaaiws-16.
pdfTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–23–Strategy4:EnsuretheSafetyandSecurityofAISystems2019UpdateCreatingrobustandtrustworthyAIsystemsSincethe2016releaseoftheNationalAIR&DStrategicPlan,therehasbeenrapidgrowthinscientificandsocietalunderstandingofAIsecurityandsafety.
Muchofthisnewknowledgehashelpedidentifynewproblems:itismoreevidentnowhowAIsystemscanbemadetodothewrongthing,learnthewrongthing,orrevealthewrongthing,forexample,throughadversarialexamples,datapoisoning,andmodelinversion,respectively.
Unfortunately,technicalsolutionsfortheseAIsafetyandsecurityproblemsremainelusive.
Toaddressalloftheseproblems,thesafetyandsecurityofAIsystemsmustbeconsideredinallstagesoftheAIsystemlifecycle,fromtheinitialdesignanddata/modelbuilding,toverificationandvalidation,deployment,operation,andmonitoring.
Indeed,thenotionof"safety(orsecurity)bydesign"mightimpartanincorrectnotionthattheseareonlyconcernsofsystemdesigners;instead,theymustbeconsideredthroughoutthesystemlifecycle,notjustatthedesignstage,andsomustbeanimportantpartoftheAIR&Dportfolio.
WhenAIcomponentsareconnectedtoothersystemsorinformationthatmustbesafeorsecure,theAIvulnerabilitiesandperformancerequirements(e.
g.
,verylowfalse-positiveandfalse-negativerates,whenoperatingoverhighvolumesofdata)AIsafetyandsecurity:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,anumberofagencieshaveinitiatedeffortssupportingStrategy4:DOTpublishednewFederalguidanceforautomatedvehiclesinOctober2018supportingthesafeintegrationofautomationintothebroadmultimodalsurfacetransportationsystem.
PreparingfortheFutureofTransportation:AutomatedVehicles3.
072advancesDOT'sprinciplesforsafeintegrationofautomatedvehicles.
Thedocumentalsoreiteratespriorsafetyguidance,providesnewmultimodalsafetyguidance,andoutlinesaprocessforworkingwithDOTasthisnewtechnologyevolves.
AsofMay2019,fourteencompanieshadreleasedVoluntarySafetySelf-Assessmentsdetailinghowtheywillincorporatesafetyintotheirdesignandtestingofautomateddrivingsystems.
73InDecember2018,IARPAannouncedtwoprogramsonAIsecurity:Secure,Assured,IntelligentLearningSystems(SAILS)74andTrojansinArtificialIntelligence(TrojAI).
75DARPAannouncedanotherprograminFebruary2019,GuaranteeingAIRobustnessagainstDeception(GARD).
76Together,theseprogramsaimtocombatarangeofattacksonAIsystems.
AsnotedinStrategy3,DoDiscommittedto"leadinginmilitaryethicsandAIsafety"asoneoffivekeyactionsoutlinedinthestrategicapproachthatguidesitseffortstoacceleratetheadoptionofAIsystems.
7772https://www.
transportation.
gov/av/373https://www.
nhtsa.
gov/automated-driving-systems/voluntary-safety-self-assessment74https://www.
iarpa.
gov/index.
php/research-programs/sails75https://www.
iarpa.
gov/index.
php/research-programs/trojai76https://www.
darpa.
mil/news-events/2019-02-0677"Summaryofthe2018DepartmentofDefenseArtificialIntelligenceStrategy":https://media.
defense.
gov/2019/Feb/12/2002088963/-1/-1/1/SUMMARY-OF-DOD-AI-STRATEGY.
PDF.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–24–areinheritedbythelargersystems.
Thesechallengesarenotstatic;asAIsystemscontinuetogrowincapabilities,theywilllikelygrowincomplexity,makingiteverharderforcorrectperformanceorprivacyofinformationtobeverifiedandvalidated.
Thiscomplexitymayalsomakeitincreasinglydifficulttoexplaindecisionsinwaysthatjustifyhighlevelsoftrustfromhumanusers(seeStrategy3).
MakingAItrustworthy—nowandintothefuture—isacriticalissuethatrequiresFederalGovernmentR&Dinvestments(seesidebar),alongwithcollaborativeeffortsamonggovernment,industry,academia,andcivilsociety.
EngineeringtrustworthyAIsystemsmaybenefitfromborrowingexistingpracticesinsafetyengineeringinotherfieldsthathavelearnedhowtoaccountforpotentialmisbehaviorofnon-AIautonomousorsemi-autonomoussystems.
However,AI-specificproblemsmeanthatnoveltechniquesforprogramanalysis,testing,formalverification,andsynthesiswillbecriticaltoestablishthatanAI-basedsystemmeetsitsspecifications—thatis,thatthesystemdoesexactlywhatitissupposedtodoandnomore.
TheseproblemsareexacerbatedinAI-basedsystemsthatcanbeeasilyfooled,evaded,andmisledinwaysthatcanhaveprofoundsecurityimplications.
AnemergingresearchareaisadversarialML,whichexploresboththeanalysisofvulnerabilitiesinMLalgorithmsaswellasalgorithmictechniquesthatyieldmorerobustlearning.
Well-knownattacksonMLincludeadversarialclassifierevasionattacks,wheretheattackerchangesbehaviortoescapebeingdetected,andpoisoningattacks,wheretrainingdataitselfiscorrupted.
ThereisgrowingneedforresearchthatsystematicallyexploresthespaceofadversariesthatattackMLandotherAI-basedsystemsandtodesignalgorithmsthatprovideprovablerobustnessguaranteesagainstclassesofadversaries.
Methodsmustbedevelopedtomakesafeandsecurethecreation,evaluation,deployment,andcontainmentofAI,andthesemethodsmustscaletomatchthecapabilityandcomplexityofAI.
Evaluatingthesemethodswillrequirenewmetrics,controlframeworks,andbenchmarksfortestingandassessingthesafetyofincreasinglypowerfulsystems.
Bothmethodsandmetricsmustincorporatehumanfactors,withsafeAIobjectivesdefinedbyhumandesigners'goals,safeAIoperationsdefinedbyhumanusers'habits,andsafeAImetricsdefinedbyhumanevaluators'understanding.
Producinghuman-drivenandhuman-understandablemethodsandmetricsforthesafetyofAIsystemswillenablepolicymakers,theprivatesector,andthepublictoaccuratelyjudgetheevolvingAIsafetylandscapeandappropriatelyproceedwithinit.
BeforeanAIsystemisputintowidespreaduse,assuranceisneededthatthesystemwilloperatesafelyandsecurely,inacontrolledmanner.
ResearchisneededtoaddressthischallengeofcreatingAIsystemsthatarereliable,dependable,andtrustworthy.
Aswithothercomplexsystems,AIsystemsfaceimportantsafetyandsecuritychallengesdueto:78Complexanduncertainenvironments:Inmanycases,AIsystemsaredesignedtooperateincomplexenvironments,withalargenumberofpotentialstatesthatcannotbeexhaustivelyexaminedortested.
Asystemmayconfrontconditionsthatwereneverconsideredduringitsdesign.
Emergentbehavior:ForAIsystemsthatlearnafterdeployment,asystem'sbehaviormaybedeterminedlargelybyperiodsoflearningunderunsupervisedconditions.
Undersuchconditions,itmaybedifficulttopredictasystem'sbehavior.
Goalmisspecification:Duetothedifficultyoftranslatinghumangoalsintocomputerinstructions,thegoalsthatareprogrammedforanAIsystemmaynotmatchthegoalsthatwereintendedbytheprogrammer.
78J.
Bornstein,"DoDAutonomyRoadmap–AutonomyCommunityofInterest,"PresentationatNDIA16thAnnualScience&EngineeringTechnologyConference,March2015.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–25–Human-machineinteractions:Inmanycases,theperformanceofanAIsystemissubstantiallyaffectedbyhumaninteractions.
Inthesecases,variationinhumanresponsesmayaffectthesafetyofthesystem.
79Toaddresstheseissuesandothers,additionalinvestmentsareneededtoadvanceAIsafetyandsecurity,80includingexplainabilityandtransparency,trust,verificationandvalidation,securityagainstattacks,andlong-termAIsafetyandvalue-alignment.
ImprovingexplainabilityandtransparencyAkeyresearchchallengeisincreasingthe"explainability"or"transparency"ofAI.
Manyalgorithms,includingthosebasedondeeplearning,areopaquetousers,withfewexistingmechanismsforexplainingtheirresults.
Thisisespeciallyproblematicfordomainssuchashealthcare,wheredoctorsneedexplanationstojustifyaparticulardiagnosisoracourseoftreatment.
AItechniquessuchasdecision-treeinductionprovidebuilt-inexplanationsbutaregenerallylessaccurate.
Thus,researchersmustdevelopsystemsthataretransparent,andintrinsicallycapableofexplainingthereasonsfortheirresultstousers.
BuildingtrustToachievetrust,AIsystemdesignersneedtocreateaccurate,reliablesystemswithinformative,user-friendlyinterfaces,whiletheoperatorsmusttakethetimeforadequatetrainingtounderstandsystemoperationandlimitsofperformance.
Complexsystemsthatarewidelytrustedbyusers,suchasmanualcontrolsforvehicles,tendtobetransparent(thesystemoperatesinamannerthatisvisibletotheuser),credible(thesystem'soutputsareacceptedbytheuser),auditable(thesystemcanbeevaluated),reliable(thesystemactsastheuserintended),andrecoverable(theusercanrecovercontrolwhendesired).
AsignificantchallengetocurrentandfutureAIsystemsremainstheinconsistentqualityofsoftwareproductiontechnology.
AsadvancesbringgreaterlinkagesbetweenhumansandAIsystems,thechallengeintheareaoftrustistokeeppacewithchangingandincreasingcapabilities,anticipatetechnologicaladvancesinadoptionandlong-termuse,andestablishgoverningprinciplesandpoliciesforthestudyofbestpracticesfordesign,construction,anduse,includingproperoperatortrainingforsafeoperation.
EnhancingverificationandvalidationNewmethodsareneededforverificationandvalidationofAIsystems.
"Verification"establishesthatasystemmeetsformalspecifications,while"validation"establishesthatasystemmeetstheuser'soperationalneeds.
SafeAIsystemsmayrequirenewmeansofassessment(determiningifthesystemismalfunctioning,perhapswhenoperatingoutsideexpectedparameters),diagnosis(determiningthecausesforthemalfunction),andrepair(adjustingthesystemtoaddressthemalfunction).
Forsystemsoperatingautonomouslyoverextendedperiodsoftime,systemdesignersmaynothaveconsideredeveryconditionthesystemwillencounter.
Suchsystemsmayneedtopossesscapabilitiesforself-assessment,self-diagnosis,andself-repairinordertoberobustandreliable.
79J.
M.
Bradshaw,R.
R.
Hoffman,M.
Johnson,andD.
D.
Woods,"TheSevenDeadlyMythsofAutonomousSystems,"IEEEIntelligentSystems28(3)(2013):54-61.
80See,forinstance:D.
Amodei,C.
Olah,J.
Steinhardt,P.
Christiano,J.
Schulman,andD.
Mane,"ConcreteProblemsinAISafety,"2016,arXiv:1606.
06565v2;S.
Russell,D.
Dewey,andM.
Tegmark,"ResearchPrioritiesforRobustandBeneficialArtificialIntelligence,"2016,arXiv:1602.
03506;T.
G.
DietterichandE.
J.
Horvitz,"RiseofConcernsaboutAI:ReflectionsandDirections,"CommunicationsoftheACM,58(10)(2015);andK.
SotalaandR.
Yampolskiy,"ResponsestocatastrophicAGIrisk:Asurvey,"PhysicaScripta,90(1),19December2014.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–26–SecuringagainstattacksAIembeddedincriticalsystemsmustberobustinordertohandleaccidentsbutshouldalsobesecuretoawiderangeofintentionalcyberattacks.
Securityengineeringinvolvesunderstandingthevulnerabilitiesofasystemandtheactionsofactorswhomaybeinterestedinattackingit.
WhilecybersecurityR&DneedsareaddressedingreaterdetailintheNITRD2016FederalCybersecurityR&DStrategicPlan,81somecybersecurityrisksarespecifictoAIsystems.
Forexample,onekeyresearchareais"adversarialmachinelearning"thatexploresthedegreetowhichAIsystemscanbecompromisedby"contaminating"trainingdata,bymodifyingalgorithms,orbymakingsubtlechangestoanobjectthatpreventitfrombeingcorrectlyidentified(e.
g.
,prostheticsthatspooffacialrecognitionsystems).
TheimplementationofAIincybersecuritysystemsthatrequireahighdegreeofautonomyisalsoanareaforfurtherstudy.
OnerecentexampleofworkinthisareaisDARPA'sCyberGrandChallengethatinvolvedAIagentsautonomouslyanalyzingandcounteringcyberattacks.
82Achievinglong-termAIsafetyandvalue-alignmentAIsystemsmayeventuallybecomecapableof"recursiveself-improvement,"inwhichsubstantialsoftwaremodificationsaremadebythesoftwareitself,ratherthanbyhumanprogrammers.
Toensurethesafetyofself-modifyingsystems,additionalresearchiscalledfortodevelop:self-monitoringarchitecturesthatchecksystemsforbehavioralconsistencywiththeoriginalgoalsofhumandesigners;confinementstrategiesforpreventingthereleaseofsystemswhiletheyarebeingevaluated;valuelearning,inwhichthevalues,goals,orintentionsofuserscanbeinferredbyasystem;andvalueframeworksthatareprovablyresistanttoself-modification.
81https://www.
nitrd.
gov/pubs/2016-Federal-Cybersecurity-Research-and-Development-Strategic-Plan.
pdf;thisisbeingupdatedin2019.
82https://archive.
darpa.
mil/CyberGrandChallenge_CompetitorSite/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–27–Strategy5:DevelopSharedPublicDatasetsandEnvironmentsforAITrainingandTesting2019UpdateIncreasingaccesstodatasetsandassociatedchallengesAtthetimeofthe2016NationalAIR&DStrategicPlan'srelease,publiclyavailabledatasetsandenvironmentswerealreadyplayingacriticalroleinpushingforwardAIR&D,particularlyinareassuchascomputervision,naturallanguageprocessing,andspeechrecognition.
ImageNet,84withmorethan14millionlabeledobjects,alongwithassociatedcomputervisioncommunitychallenges(e.
g.
,theImageNetLargeScaleVisualRecognitionChallenge85thatevaluatesalgorithmsforobjectdetectionandimageclassification),haveplayedanespeciallyvitalroleinthecommunity.
AstranslationalapplicationsforMLarebeingfoundinmyriadapplicationareassuchashealthcare,medicine,andsmartandconnectedcommunities,theneedhasgrownforpubliclyavailabledatasetsindomain-specificareas.
Theimportanceofdatasetsandmodels–inparticular,thoseoftheFederalGovernment–isexplicitlycalledoutinthe2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligence:1HeadsofallagenciesshallreviewtheirFederaldataandmodelstoidentifyopportunitiestoincreaseaccessandusebythegreaternon-FederalAIresearchcommunityinamannerthatbenefitsthatcommunity,whileprotectingsafety,security,privacy,andconfidentiality.
Specifically,agenciesshallimprovedataandmodelinventorydocumentationtoenablediscoveryandusability,andshallSharedPublicDatasetsandEnvironmentsforAITrainingandTesting:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,anumberofagencieshaveinitiatedeffortssupportingStrategy5:DOTsponsoredtheSecondStrategicHighwayResearchProgram(SHRP2)NaturalisticDrivingStudy(NDS),83whichrecordedmorethan5.
4milliontripstakenbymorethan3,400driversandvehicles.
Anin-vehicledataacquisitionsystem(DAS)unitgatheredandstoreddatafromforwardradar,fourvideocameras,accelerometers,vehiclenetworkinformation,ageographicpositioningsystem,andanonboardlanetracker.
DatafromtheDASwererecordedcontinuouslywhileparticipants'vehicleswereoperating.
WhereassummariesoftheNDSdataarepublic,accesstothedetaileddatasetsrequiresqualifiedresearchethicstraining.
TheVADataCommonsiscreatingthelargestlinkedmedical–genomicsdatasetintheworldwithtoolsforenablingMLandAI,andguidedbyveterans'preferences.
ThiseffortisleveragingapplicableNISTstandards,laws,andexecutiveorders.
GSA(GeneralServicesAdministration)isworkingtoenabletheuseofcloudcomputingresourcesforfederallyfundedAIR&D.
Data.
govandcode.
gov,housedatGSA,containover246,000datasetsandcodefromacrossagenciesandautomaticallyharvestdatasetsreleasedbyagencies.
TheNIHScienceandTechnologyResearchInfrastructureforDiscovery,Experimentation,andSustainability(STRIDES)initiative,apartnershipwithindustry-leadingcloudserviceproviders,isenablingresearcheraccesstomajordataassetsthatarefundedacrossNIHandthatarestoredincloudenvironments.
83https://insight.
shrp2nds.
us/84http://www.
image-net.
org/85http://www.
image-net.
org/challenges/LSVRC/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–28–prioritizeimprovementstoaccessandqualityofAIdataandmodelsbasedontheAIresearchcommunity'sfeedback.
AnewNSTCSubcommitteeonOpenSciencewascreatedin2018tocoordinateFederaleffortsonopenandFAIR(findable,accessible,interoperable,andreusable)data.
R&Dinvestmentswillbeneededtodeveloptoolsandresourcesthatmakeiteasiertoidentify,use,andmanipulaterelevantdatasets(includingFederaldatasets),verifydataprovenance,andrespectappropriateusepolicy.
ManyofthesedatasetsthemselvesmaybeoflimiteduseinanAIcontextwithoutaninvestmentinlabelingandcuration.
FederalagenciesshouldengageandworkwithAIstakeholderstoensurethatappropriatelyvetteddatasetsandmodelsthatarereleasedforsharingarereadyandfitforuseandthattheyaremaintainedasstandardsandnormsevolve.
Ultimately,developmentandadoptionofbestpracticesandstandardsindocumentingdatasetandmodelprovenancewillenhancetrustworthinessandresponsibleuseofAItechnologies.
Since2016,therehavealsobeenincreasedconcernsaboutdatacontent,suchaspotentialbias(seeStrategy3)86,87orprivateinformationleakage.
The2016NationalAIR&DStrategicPlannotedthat"datasetdevelopmentandsharingmust…followapplicablelawsandregulationsandbecarriedoutinanethicalmanner.
"TheDOT-supportedInSightprojectprovidessuchcarefullystructuredaccesstodatacollectedduringtheNaturalisticDrivingStudy(seesidebar).
The2016NationalAIR&DStrategicPlanalsonotedthatnew"technologiesareneededtoensuresafesharingofdata,sincedataownerstakeonriskwhensharingtheirdatawiththeresearchcommunity.
"Forexample,CryptoNets88allowsneuralnetworkstooperateoverencrypteddata,ensuringthatdataremainconfidential,becausedecryptionkeysarenotneededinneuralnetworks.
ResearchershavealsobegundevelopingnewMLtechniquesthatuseadifferentialprivacyframeworktoprovidequantifiableprivacyguaranteesovertheuseddata.
89Atthesametime,privacymethodsmustremainsufficientlyexplainableandtransparenttohelpresearcherscorrectthemandmakethemsafe,efficient,andaccurate.
Furthermore,AIcouldrevealdiscoveriesbeyondtheoriginalorintendedscope;therefore,researchersmustremaincognizantofthepotentialdangersinaccesstodataordiscoveriesbyadversaries.
Dataaloneareoflittleusewithouttheabilitytobringcomputationalresourcestobearonlarge-scalepublicdatasets.
TheimportanceofcomputationalresourcestoAIR&Discalledoutinthe2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligence:1TheSecretariesofDefense,Commerce,HealthandHumanServices,andEnergy,theAdministratoroftheNationalAeronauticsandSpaceAdministration,andtheDirectoroftheNationalScienceFoundationshall,totheextentappropriateandconsistentwithapplicablelaw,prioritizetheallocationofhigh-performancecomputingresourcesforAI-relatedapplicationsthrough:(i)increasedassignmentofdiscretionaryallocationofresourcesandresourcereserves;or(ii)anyotherappropriatemechanisms.
86EmilyM.
BenderandBatyaFriedman,"DataStatementsforNLP:TowardMitigatingSystemBiasandEnablingBetterScience,"TransactionsoftheAssociationforComputationalLinguistics6(2018):587-604.
87AylinCaliskan,JoannaJ.
Bryson,andArvindNarayanan,"Semanticsderivedautomaticallyfromlanguagecorporacontainhuman-likebiases,"Science356(6334):183-186,14Apr2017.
88RanGilad-Bachrach,NathanDowlin,KimLaine,KristinLauter,MichaelNaehrig,JohnWernsing,"CryptoNets:Applyingneuralnetworkstoencrypteddatawithhighthroughputandaccuracy,"2016InternationalConferenceonMachineLearning48:201-210;http://proceedings.
mlr.
press/v48/.
89MartinAbadi,AndyChu,IanGoodfellow,H.
BrendanMcMahan,IlyaMironov,KunalTalwar,andLiZhang,"DeepLearningwithDifferentialPrivacy,"23rdACMConferenceonComputerandCommunicationsSecurity,"2016:308-318.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–29–and:…theSelectCommittee,incoordinationwiththeGeneralServicesAdministration(GSA),shallsubmitareporttothePresidentmakingrecommendationsonbetterenablingtheuseofcloudcomputingresourcesforfederallyfundedAIR&D.
TheneedforcomputationalcapacityformanyAIchallengeshasbeenincreasingrapidly.
32FederalfundingmayprovidecomputationalcapabilitiesforFederally-fundedresearch.
Somecompaniesanduniversities,however,mayhaveadditionalcomputationaldemands.
Overall,thereisanationalneedtostudyandinvestinsharedcomputationalresourcestopromoteAIR&D.
ThebenefitsofAIwillcontinuetoaccrue,butonlytotheextentthattrainingandtestingresourcesforAIaredevelopedandmadeavailable.
Thevariety,depth,quality,andaccuracyoftrainingdatasetsandotherresourcessignificantlyaffectsAIperformance.
ManydifferentAItechnologiesrequirehigh-qualitydatafortrainingandtesting,aswellasdynamic,interactivetestbedsandsimulationenvironments.
Morethanjustatechnicalquestion,thisisasignificant"publicgood"challenge,asprogresswouldsufferifAItrainingandtestingislimitedtoonlyafewentitiesthatalreadyholdvaluabledatasetsandresources,yetwemustsimultaneouslyrespectcommercialandindividualrightsandinterestsinthedata.
Researchisneededtodevelophigh-qualitydatasetsandenvironmentsforawidevarietyofAIapplicationsandtoenableresponsibleaccesstogooddatasetsandtestingandtrainingresources.
Additionalopen-sourcesoftwarelibrariesandtoolkitsarealsoneededtoacceleratetheadvancementofAIR&D.
Thefollowingsubsectionsoutlinethesekeyareasofimportance.
DevelopingandmakingaccessibleawidevarietyofdatasetstomeettheneedsofadiversespectrumofAIinterestsandapplicationsTheintegrityandavailabilityofAItrainingandtestingdatasetsiscrucialtoensuringscientificallyreliableresults.
Thetechnicalaswellasthesocio-technicalinfrastructurenecessarytosupportreproducibleresearchinthedigitalareahasbeenrecognizedasanimportantchallenge—andisessentialtoAItechnologiesaswell.
ThelackofvettedandopenlyavailabledatasetswithidentifiedprovenancetoenablereproducibilityisacriticalfactortoconfidentadvancementinAI.
90Asinotherdata-intensivesciences,capturingdataprovenanceiscritical.
Researchersmustbeabletoreproduceresultswiththesameaswellasdifferentdatasets.
Datasetsmustberepresentativeofchallengingreal-worldapplications,andnotjustsimplifiedversions.
Tomakeprogressquickly,emphasisshouldbeplacedonmakingavailablealreadyexistingdatasetsheldbygovernment,thosethatcanbedevelopedwithFederalfunding,and,totheextentpossible,thoseheldbyindustry.
ThemachinelearningaspectoftheAIchallengeisoftenlinkedwith"bigdata"analysis.
Consideringthewidevarietyofrelevantdatasets,itremainsagrowingchallengetohaveappropriaterepresentation,access,andanalysisofunstructuredorsemi-structureddata.
Howcanthedataberepresented—inabsoluteaswellasrelative(context-dependent)termsCurrentreal-worlddatabasescanbehighlysusceptibletoinconsistent,incomplete,andnoisydata.
Therefore,anumberofdatapreprocessingtechniques(e.
g.
,datacleaning,integration,transformation,reduction,andrepresentation)areimportanttoestablishingusefuldatasetsforAIapplications.
Howdoesthedatapreprocessingimpactdataquality,especiallywhenadditionalanalysisisperformed90Towardthisend,in2016theIntelligenceAdvancedResearchProjectsActivityissuedaRequestforInformationonnoveltrainingdatasetsandenvironmentstoadvanceAI.
Seehttps://iarpa.
gov/index.
php/working-with-iarpa/requests-for-information/novel-training-datasets-and-environments-to-advance-artificial-intelligence.

THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–30–EncouragingthesharingofAIdatasets—especiallyforgovernment-fundedresearch—wouldlikelystimulateinnovativeAIapproachesandsolutions.
However,technologiesareneededtoensuresafesharingofdata,sincedataownerstakeonriskwhensharingtheirdatawiththeresearchcommunity.
Datasetdevelopmentandsharingmustalsofollowapplicablelawsandregulationsandbecarriedoutinanethicalmanner.
Riskscanariseinvariousways:inappropriateuseofdatasets,inaccurateorinappropriatedisclosure,andlimitationsindatade-identificationtechniquestoensureprivacyandconfidentialityprotections.
MakingtrainingandtestingresourcesresponsivetocommercialandpublicinterestsWiththecontinuingexplosionofdata,datasources,andinformationtechnologyworldwide,boththenumberandsizeofdatasetsareincreasing.
Thetechniquesandtechnologiestoanalyzedataarenotkeepingupwiththehighvolumeofrawinformationsources.
Datacapture,curation,analysis,andvisualizationareallkeyresearchchallenges,andthescienceneededtoextractvaluableknowledgefromenormousamountsofdataislaggingbehind.
Whiledatarepositoriesexist,theyareoftenunabletodealwiththescalingupofdatasets,havelimiteddataprovenanceinformation,anddonotsupportsemanticallyrichdatasearches.
Dynamic,agilerepositoriesareneeded.
Oneexampleofthekindofopen/sharinginfrastructureprogramthatisneededtosupporttheneedsofAIresearchistheIMPACTprogram(InformationMarketplaceforPolicyandAnalysisofCyber-risk&Trust)developedbytheDepartmentofHomelandSecurity(DHS).
91Thisprogramsupportstheglobalcybersecurityriskresearcheffortbycoordinatinganddevelopingreal-worlddataandinformationsharingcapabilities,includingtools,models,andmethodologies.
IMPACTalsosupportsempiricaldatasharingbetweentheinternationalcybersecurityR&Dcommunity,criticalinfrastructureproviders,andtheirgovernmentsupporters.
AIR&DwouldbenefitfromcomparableprogramsacrossallAIapplications.
Developingopen-sourcesoftwarelibrariesandtoolkitsTheincreasedavailabilityofopen-sourcesoftwarelibrariesandtoolkitsprovidesaccesstocutting-edgeAItechnologiesforanydeveloperwithanInternetconnection.
ResourcessuchastheWekatoolkit,92MALLET,93andOpenNLP,94amongmanyothers,haveacceleratedthedevelopmentandapplicationofAI.
Developmenttools,includingfreeorlow-costcoderepositoryandversioncontrolsystems,aswellasfreeorlow-costdevelopmentlanguages(e.
g.
,R,Octave,andPython)providelowbarrierstousingandextendingtheselibraries.
Inaddition,forthosewhomaynotwanttointegratetheselibrariesdirectly,anynumberofcloud-basedmachinelearningservicesexistthatcanperformtaskssuchasimageclassificationondemandthroughlow-latencywebprotocolsthatrequirelittleornoprogrammingforuse.
Finally,manyofthesewebservicesalsooffertheuseofspecializedhardware,includingGPU-basedsystems.
ItisreasonabletoassumethatspecializedhardwareforAIalgorithms,includingneuromorphicprocessors,willalsobecomewidelyavailablethroughtheseservices.
Together,theseresourcesprovideanAItechnologyinfrastructurethatencouragesmarketplaceinnovationbyallowingentrepreneurstodevelopsolutionsthatsolvenarrowdomainproblemswithoutrequiringexpensivehardwareorsoftware,withoutrequiringahighlevelofAIexpertise,andpermittingrapidscaling-upofsystemsondemand.
FornarrowAIdomains,barrierstomarketplaceinnovationareextremelylowrelativetomanyothertechnologyareas.
91https://www.
dhs.
gov/csd-impact92https://sourceforge.
net/projects/weka/93http://mallet.
cs.
umass.
edu94https://opennlp.
apache.
orgTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–31–Tohelpsupportacontinuedhighlevelofinnovationinthisarea,theU.
S.
Governmentcanboosteffortsinthedevelopment,support,anduseofopenAItechnologies.
Particularlybeneficialwouldbeopenresourcesthatusestandardizedoropenformatsandopenstandardsforrepresentingsemanticinformation,includingdomainontologieswhenavailable.
GovernmentmayalsoencouragegreateradoptionofopenAIresourcesbyacceleratingtheuseofopenAItechnologieswithinthegovernmentitself,andthushelptomaintainalowbarriertoentryforinnovators.
Wheneverpossible,governmentshouldcontributealgorithmsandsoftwaretoopensourceprojects.
Becausegovernmenthasspecificconcerns,suchasagreateremphasisondataprivacyandsecurity,itmaybenecessaryforthegovernmenttodevelopmechanismstoeasegovernmentadoptionofAIsystems.
Forexample,itmaybeusefultocreateataskforcethatcanperforma"horizonscan"acrossgovernmentagenciestofindparticularAIapplicationareaswithindepartments,andthendeterminespecificconcernsthatwouldneedtobeaddressedtopermitadoptionofsuchtechniquesbytheseagencies.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–32–Strategy6:MeasureandEvaluateAITechnologiesthroughStandardsandBenchmarks2019UpdateSupportingdevelopmentofAItechnicalstandardsandrelatedtoolsThe2016NationalAIR&DStrategicPlanstatesthat"Standards,benchmarks,testbeds,andtheiradoptionbytheAIcommunityareessentialforguidingandpromotingR&DofAItechnologies.
"Intheinterveningthreeyears,emphasisonstandardsandbenchmarkshascontinuedtoriseintheU.
S.
andglobally.
The2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligenceexplicitlycallsouttheimportanceofsuchstandards:1…[T]heSecretaryofCommerce,throughtheDirectorof[NIST],shallissueaplanforFederalengagementinthedevelopmentoftechnicalstandardsandrelatedtoolsinsupportofreliable,robust,andtrustworthysystemsthatuseAItechnologies.
WithAIinnovationpotentiallyimpactingallsectorsanddomainsofsociety,manystandardsdevelopmentorganizationshavenewAI-relatedconsiderationsandworkitemsunderway,includingactivitiesrelatedtoAIethicsandtrustworthyAIsystems(seeStrategy3).
TheInternationalOrganizationforStandardization(ISO)andtheInternationalElectrotechnicalCommission(IEC)haveconvenedajointtechnicalsubcommitteeonAI(ISO/IECJointTechnicalCommittee1,Subcommittee42onArtificialIntelligence95)todevelopstandardsforAIsystemsandassociatedconsiderations.
ItiscriticalthatFederal,industry,andacademicresearcherscontinuetoinformtheseactivities,particularlyasAIadvancesandsystemsreachintoareassuchastransportation,healthcare,andfoodthatalignwiththemissionsofgovernmentagencies.
Since2016,thesurgeinAI-relatedstandardsactivitieshasoutpacedthelaunchofnewAI-focusedbenchmarksandevaluations,particularlyasrelatedtotrustworthinessofAIsystems.
IntheStandards,benchmarks,andrelatedtools:RecentagencyR&DprogramsSincethereleaseofthe2016NationalAIR&DStrategicPlan,NISTinparticularhasinitiatedeffortsforStrategy6:NISTisengagedinthestandardizationprogramofISO/IECJTC1SC42onArtificialIntelligence.
95ANISTexpertistheconvenerfortheBigDataworkeffortinSC42.
TheU.
S.
delegationtoSC42includesNISTandotherFederalagencyexperts,aswellasrepresentativesfromindustryandacademia.
U.
S.
inputtoSC42isfacilitatedbytheInternationalCommitteeforInformationTechnologyStandards(INCITS).
NISTstaffparticipateinadditionalAIstandardsactivitiesthroughstandardsorganizations,suchastheAmericanSocietyofMechanicalEngineers,IEEE,andISO/IEC.
Theiractivitiescoversuchtopicsascomputationalmodelingforadvancedmanufacturing,ontologiesforroboticsandautomation,personaldataprivacy,andalgorithmicbias.
NISTexpertsareraisingawarenessabouttheimportanceofconsensusstandardsforAIinmultilateralfora,includingbodiessuchasG20andG7.
96NISTbringsuniqueFederalGovernmentexpertisethatgroundspolicydiscussionsinpractice,inparticular,throughclosecollaborationwiththeprivatesector.
Similarly,NISTlendsitsstandardsandrelatedexperiencetointergovernmentalbilateraldiscussions.
95https://www.
iso.
org/committee/6794475.
html96https://home.
treasury.
gov/policy-issues/international/g-7-and-g-20THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–33–interveningtime,however,considerationsoffairnessandbiasinbenchmarkdatasetshavebecomeincreasinglyimportant,withresearcherspursuingnewfacialrecognitiondatasetsthatseektominimizebias.
Muchmoreplentifularebenchmarksthattesttheapplication-levelperformanceofAIalgorithms(e.
g.
,false-positiveorfalse-negativeratesforclassificationalgorithms)andbenchmarksthatquantifythecompute-levelperformanceofAIsoftwareandhardwaresystems.
TwosuchrecentactivitiesareMLPerf97andDAWNbench.
98Assessing,promoting,andassuringallaspectsofAItrustworthinessrequiresmeasuringandevaluatingAItechnologyperformancethroughbenchmarksandstandards.
Beyondbeingsafe,secure,reliable,resilient,explainable,andtransparent,trustworthyAImustpreserveprivacywhiledetectingandavoidinginappropriatebias.
AsAItechnologiesevolve,sowilltheneedtodevelopnewmetricsandtestingrequirementsforvalidationoftheseessentialcharacteristics.
Standards,benchmarks,testbeds,andtheiradoptionbytheAIcommunityareessentialforguidingandpromotingR&DofAItechnologies.
Thefollowingsubsectionsoutlineareaswhereadditionalprogressmustbemade.
DevelopingabroadspectrumofAIstandardsThedevelopmentofstandardsmustbehastenedtokeeppacewiththerapidlyevolvingcapabilitiesandexpandingdomainsofAIapplications.
Standardsproviderequirements,specifications,guidelines,orcharacteristicsthatcanbeusedconsistentlytoensurethatAItechnologiesmeetcriticalobjectivesforfunctionalityandinteroperability,andthattheyperformreliablyandsafely.
Adoptionofstandardsbringscredibilitytotechnologyadvancementsandfacilitatesanexpandedinteroperablemarketplace.
OneexampleofanAI-relevantstandardthathasbeendevelopedisP1872-2015(StandardOntologiesforRoboticsandAutomation),developedbytheInstituteofElectricalandElectronicsEngineers.
Thisstandardprovidesasystematicwayofrepresentingknowledgeandacommonsetoftermsanddefinitions.
Theseallowforunambiguousknowledgetransferamonghumans,robots,andotherartificialsystems,aswellasprovideafoundationalbasisfortheapplicationofAItechnologiestorobotics.
AdditionalworkinAIstandardsdevelopmentisneededacrossallsubdomainsofAI.
Standardsareneededtoaddress:Softwareengineering:tomanagesystemcomplexity,sustainment,security,andtomonitorandcontrolemergentbehaviors;Performance:toensureaccuracy,reliability,robustness,accessibility,andscalability;Metrics:toquantifyfactorsimpactingperformanceandcompliancetostandards;Safety:toevaluateriskmanagementandhazardanalysisofsystems,humancomputerinteractions,controlsystems,andregulatorycompliance;Usability:toensurethatinterfacesandcontrolsareeffective,efficient,andintuitive;Interoperability:todefineinterchangeablecomponents,data,andtransactionmodelsviastandardandcompatibleinterfaces;Security:toaddresstheconfidentiality,integrity,andavailabilityofinformation,aswellascybersecurity;Privacy:tocontrolfortheprotectionofinformationwhilebeingprocessed,whenintransit,orbeingstored;97https://mlperf.
org/98https://dawn.
cs.
stanford.
edu/benchmark/THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–34–Traceability:toprovidearecordofevents(theirimplementation,testing,andcompletion),andforthecurationofdata;andDomains:todefinedomain-specificstandardlexiconsandcorrespondingframeworks.
EstablishingAItechnologybenchmarksBenchmarks,madeupoftestsandevaluations,providequantitativemeasuresfordevelopingstandardsandassessingcompliancetostandards.
Benchmarksdriveinnovationbypromotingadvancementsaimedataddressingstrategicallyselectedscenarios;theyadditionallyprovideobjectivedatatotracktheevolutionofAIscienceandtechnologies.
ToeffectivelyevaluateAItechnologies,relevantandeffectivetestingmethodologiesandmetricsmustbedevelopedandstandardized.
Standardtestingmethodswillprescribeprotocolsandproceduresforassessing,comparing,andmanagingtheperformanceofAItechnologies.
StandardmetricsareneededtodefinequantifiablemeasuresinordertocharacterizeAItechnologies,includingbutnotlimitedto:accuracy,complexity,trustandcompetency,riskanduncertainty,explainability,unintendedbias,comparisontohumanperformance,andeconomicimpact.
Itisimportanttonotethatbenchmarksaredatadriven.
Strategy5discussestheimportanceofdatasetsfortrainingandtesting.
AsasuccessfulexampleofAI-relevantbenchmarks,theNationalInstituteofStandardsandTechnologyhasdevelopedacomprehensivesetofstandardtestmethodsandassociatedperformancemetricstoassesskeycapabilitiesofemergencyresponserobots.
Theobjectiveistofacilitatequantitativecomparisonsofdifferentrobotmodelsbymakinguseofstatisticallysignificantdataonrobotcapabilitiesthatwascapturedusingthestandardtestmethods.
Thesecomparisonscanguidepurchasingdecisionsandhelpdeveloperstounderstanddeploymentcapabilities.
TheresultingtestmethodsarebeingstandardizedthoughtheASTMInternationalStandardsCommitteeonHomelandSecurityApplicationsforroboticoperationalequipment(referredtoasstandardE54.
08.
01).
99VersionsofthetestmethodsareusedtochallengetheresearchcommunitythroughtheRoboCupRescueRobotLeaguecompetitions,100whichemphasizeautonomouscapabilities.
AnotherexampleistheIEEEAgileRoboticsforIndustrialAutomationCompetition(ARIAC),101ajointeffortbetweenIEEEandNIST,102whichpromotesrobotagilitybyutilizingthelatestadvancesinartificialintelligenceandrobotplanning.
Acorefocusofthiscompetitionistotesttheagilityofindustrialrobotsystems,withthegoalofenablingthoseontheshopfloorstobemoreproductive,moreautonomous,andrequiringlesstimefromshopfloorworkers.
WhiletheseeffortsprovideastrongfoundationfordrivingAIbenchmarkingforward,theyarelimitedbybeingdomain-specific.
Additionalstandards,testbeds,andbenchmarksareneededacrossabroaderrangeofdomainstoensurethatAIsolutionsarebroadlyapplicableandwidelyadopted.
IncreasingtheavailabilityofAItestbedsTheimportanceoftestbedswasstatedintheCyberExperimentationoftheFuturereport:"Testbedsareessentialsothatresearcherscanuseactualoperationaldatatomodelandrunexperimentsonreal-worldsystem[s]…andscenariosingoodtestenvironments.
"103Havingadequatetestbedsisa992019update:TheresultingtestmethodsarenowstandardsissuedbyASTMInternationalStandardsCommitteeonHomelandSecruityApplicationsforResponseRobots(referredtoasE54.
09).
100http://www.
robocup2016.
org/en/101http://robotagility.
wixsite.
com/competition1022019update:IEEEisnolongerapartnerofARIAC,whichisnowinitsthirdyear.
103SRIInternationalandUSCInformationSciencesInstitute,"CybersecurityExperimentationoftheFuture(CEF):CatalyzingaNewGenerationofExperimentalCybersecurityResearch,"FinalReport,July31,2015.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–35–needacrossallareasofAI.
Thegovernmenthasmassiveamountsofmission-sensitivedatauniquetogovernment,butmuchofthisdatacannotbedistributedtotheoutsideresearchcommunity.
Appropriateprogramscouldbeestablishedforacademicandindustrialresearcherstoconductresearchwithinsecuredandcuratedtestbedenvironmentsestablishedbyspecificagencies.
AImodelsandexperimentalmethodscouldbesharedandvalidatedbytheresearchcommunitybyhavingaccesstothesetestenvironments,affordingAIscientists,engineers,andstudentsuniqueresearchopportunitiesnototherwiseavailable.
EngagingtheAIcommunityinstandardsandbenchmarksGovernmentleadershipandcoordinationisneededtodrivestandardizationandencourageitswidespreaduseingovernment,academia,andindustry.
TheAIcommunity—madeupofusers,industry,academia,andgovernment—mustbeenergizedtoparticipateindevelopingstandardsandbenchmarkprograms.
Aseachgovernmentagencyengagesthecommunityindifferentwaysbasedonitsroleandmission,communityinteractionscanbeleveragedthroughcoordinationinordertostrengthentheirimpact.
Thiscoordinationisneededtocollectivelygatheruser-drivenrequirements,anticipatedeveloper-drivenstandards,andpromoteeducationalopportunities.
User-drivenrequirementsshapetheobjectivesanddesignofchallengeproblemsandenabletechnologyevaluation.
HavingcommunitybenchmarksfocusesR&Dtodefineprogress,closegaps,anddriveinnovativesolutionsforspecificproblems.
Thesebenchmarksmustincludemethodsfordefiningandassigninggroundtruth.
ThecreationofbenchmarksimulationandanalysistoolswillalsoaccelerateAIdevelopments.
Theresultsofthesebenchmarksalsohelpmatchtherighttechnologytotheuser'sneed,formingobjectivecriteriaforstandardscompliance,qualifiedproductlists,andpotentialsourceselection.
IndustryandacademiaaretheprimarysourcesforemergingAItechnologies.
Promotingandcoordinatingtheirparticipationinstandardsandbenchmarkingactivitiesarecritical.
Assolutionsemerge,opportunitiesaboundforanticipatingdeveloper-anduser-drivenstandardsthroughsharingcommonvisionsfortechnicalarchitectures,developingreferenceimplementationsofemergingstandardstoshowfeasibility,andconductingprecompetitivetestingtoensurehigh-qualityandinteroperablesolutions,aswellastodevelopbestpracticesfortechnologyapplications.
Onesuccessfulexampleofahigh-impact,community-based,AI-relevantbenchmarkprogramistheTextRetrievalConference(TREC),104whichwasstartedbyNISTin1992toprovidetheinfrastructurenecessaryforlarge-scaleevaluationofinformationretrievalmethodologies.
Morethan250groupshaveparticipatedinTREC,includingacademicandcommercialorganizationsbothlargeandsmall.
Thestandard,widelyavailable,andcarefullyconstructedsetofdataputforthbyTREChasbeencreditedwithrevitalizingresearchoninformationretrieval.
105,106AsecondexampleistheNISTperiodicbenchmarkprogramintheareaofmachinevisionappliedtobiometrics,107particularlyfacerecognition.
108ThisbeganwiththeFaceRecognitionTechnology(FERET)evaluationin1993,whichprovidedastandarddatasetoffacephotosdesignedtosupportfacerecognitionalgorithmdevelopmentaswellasanevaluationprotocol.
ThisefforthasevolvedovertheyearsintotheFace104http://trec.
nist.
gov105E.
M.
VoorheesandD.
K.
Harman,TRECExperimentandEvaluationinInformationRetrieval(Cambridge:MITPress,2005).
106http://googleblog.
blogspot.
com/2008/03/why-data-matters.
html107http://biometrics.
nist.
gov108http://face.
nist.
govTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–36–RecognitionVendorTest(FRVT),109involvingthedistributionofdatasets,hostingofchallengeproblems,andconductingofsequesteredtechnologyevaluations.
Thisbenchmarkprogramhascontributedgreatlytotheimprovementoffacialrecognitiontechnology.
BothTRECandFRVTcanserveasexamplesofeffectiveAI-relevantcommunitybenchmarkingactivities,butsimilareffortsareneededinotherareasofAI.
Itisimportanttonotethatdevelopingandadoptingstandards,aswellasparticipatinginbenchmarkactivities,comeswithacost.
R&Dorganizationsareincentivizedwhentheyseesignificantbenefit.
UpdatingacquisitionprocessesacrossagenciestoincludespecificrequirementsforAIstandardsinrequestsforproposalswillencouragethecommunitytofurtherengageinstandardsdevelopmentandadoption.
Community-basedbenchmarks,suchasTRECandFRVT,alsolowerbarriersandstrengthenincentivesbyprovidingtypesoftrainingandtestingdataotherwiseinaccessible,fosteringhealthycompetitionbetweentechnologydeveloperstodrivebest-of-breedalgorithms,andprovidingobjectiveandcomparativeperformancemetricsforrelevantsourceselections.
109P.
J.
Phillips,"ImprovingFaceRecognitionTechnology,"Computer44(3)(2011):84-96.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–37–Strategy7:BetterUnderstandtheNationalAIR&DWorkforceNeeds2019UpdateAdvancingtheAIR&Dworkforce,includingthoseworkingonAIsystemsandthoseworkingalongsidethem,tosustainU.
S.
leadershipSincethereleaseofthe2016NationalAIR&DStrategicPlan,thedemandforAIresearchersandpractitionershasgrownrapidly.
Studieshaveshownthatthenumberofhiringopportunitiesisexpectedtoriseintothemillionsoverthenextdecade.
Asonedatapoint,theU.
S.
BureauofLaborStatisticsprojectsthatthenumberofpositionsforcomputerandinformationscientistsandengineerswillgrowby19%from2016to2026,almostthreetimesfasterthantheaverageforalloccupations.
111Moreover,through2028,AIresearchersareexpectedcontributetoasmuchas$11.
5trillionofcumulativegrowthpromisedbyintelligenttechnologiesintheG20countriesalone.
112U.
S.
academicinstitutionsarestrugglingtokeeppacewiththeexplosivegrowthinstudentinterestandenrollmentinAI.
113,114,115Atthesametime,industry,withitssustainedfinancialsupportandaccesstoadvancedcomputingfacilitiesanddatasets,exertsastrongpullonacademicresearchandteachingtalent.
116ItiscriticaltomaintainarobustacademicresearchecosysteminAIthat,incollaborationwithindustryR&D,cancontinuetodelivertremendousdividends117byadvancingnationalhealth,prosperity,andwelfare,andsecuringthenationaldefense.
NationalAIR&Dworkforce:RecentagencyactivitiesSincethereleaseofthe2016NationalAIR&DStrategicPlan,anumberofagencieshaveinitiatedeffortssupportingStrategy7:ApartfromsupportingundergraduateandgraduatestudentsthroughstandardAIresearchgrants,agenciesareprioritizingcomputational-anddata-enabledscienceandengineeringintheirgraduatefellowshipprograms.
Forexample,in2018,DOEaddedanewtracktoitsComputationalScienceGraduateFellowshipprogram.
Thistracksupportsstudentspursuingadvanceddegreesinappliedmathematics,statistics,orcomputerscience,andpromotesmoreeffectiveuseofhigh-performancesystems,includingintheareasofAI,ML,anddeeplearning.
44,110Alsoin2018,NSFbeganprioritizingcomputationalanddata-enabledscienceandengineeringinasubsetofawardeesofitsGraduateResearchFellowshipsProgram.
TheCensusBureauhascreatedtheStatisticalDataModernization(SDM)projecttobringitsworkforce,operations,andtechnologiesuptothecurrentstateoftheartandsetthestandardforstatisticalagenciesintoday'sdata-drivensociety.
SDM'sworkforcetransformationcomponentwillenablethehiringofnewdatascientistswithexpertiseinnewmethodsandanalytics,includingtheuseofAImethodsandtoolstoprocessandanalyzebigdata.
Theworkforcetransformationwillalsoaddresstheupskillingofthecurrentdatascienceworkforce.
110https://www.
krellinst.
org/csgf/math-cs111https://www.
bls.
gov/ooh/computer-and-information-technology/computer-and-information-research-scientists.
htm112https://www.
accenture.
com/t20180920T094705Z__w__/us-en/_acnmedia/Thought-Leadership-Assets/PDF/Accenture-Education-and-Technology-Skills-Research.
pdf113https://cra.
org/data/generation-cs/114https://cra.
org/wp-content/uploads/2018/05/2017-Taulbee-Survey-Report.
pdf115http://web.
cs.
wpi.
edu/~cew/papers/CSareas19.
pdf116https://www.
nitrd.
gov/rfi/ai/2018/AI-RFI-Response-2018-Yolanda-Gil-AAAI.
pdf117https://www.
nap.
edu/catalog/13427/continuing-innovation-in-information-technologyTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–38–Inthethreeyearssincethereleaseofthe2016NationalAIR&DStrategicPlan,variousreportshavecalledforcontinuedsupportforthedevelopmentofinstructionalmaterialsandteacherprofessionaldevelopmentincomputerscienceatalllevels.
EmphasisisneededattheK–12levelstofeedtheNation'spipelineofAIresearchersovermanydecades.
118Attheundergraduatelevel,thereisaneedtofocusonintegratingadvancedcomputationalskillsandmethodswithdomain-specificknowledgefromotherdisciplines,giventhegrowingroleofcomputingacrossdisciplines.
119Sustainedsupportisalsoneededatthegraduatelevel,wherestudentsareconductingfundamentalresearchinMLandAI.
Indeed,the2019ExecutiveOrderonMaintainingAmericanLeadershipinArtificialIntelligencerequiresthat:1Headsofimplementingagenciesthatalsoprovideeducationalgrantsshall,totheextentconsistentwithapplicablelaw,considerAIasapriorityareawithinexistingFederalfellowshipandserviceprograms…[including]…(A)highschool,undergraduate,andgraduatefellowship;alternativeeducation;andtrainingprograms;(B)programstorecognizeandfundearly-careeruniversityfacultywhoconductAIR&D,includingthroughPresidentialawardsandrecognitions;(C)scholarshipforserviceprograms;(D)directcommissioningprogramsoftheUnitedStatesArmedForces;and(E)programsthatsupportthedevelopmentofinstructionalprogramsandcurriculathatencouragetheintegrationofAItechnologiesintocoursesinordertofacilitatepersonalizedandadaptivelearningexperiencesforformalandinformaleducationandtraining.
Morebroadly,theneedforafirmgroundingincomputationalthinking,includingthroughcomputerscienceeducation,isalsonotedprominentlyintheFederalGovernment'sDecember2018five-yearstrategicplanforscience,technology,engineering,andmathematics(STEM)education.
120Inaddition,itisimperativetobroadentheparticipationamonggroupstraditionallyunderrepresentedincomputingandrelatedfields.
Finally,theAIR&Dworkforcewillconsistofmultidisciplinaryteamscomprisingnotjustcomputerandinformationscientistsandengineers,butalsoexpertsfromotherfieldskeytoAIandMLinnovationanditsapplication,includingcognitivescienceandpsychology,economicsandgametheory,engineeringandcontroltheory,ethics,linguistics,mathematics,philosophy,andthemanydomainsinwhichAImaybeapplied.
FederalagenciesaregivingprioritytotrainingandfellowshipprogramsatalllevelstopreparetheworkforcewithrequisiteAIR&Dskillsthroughapprenticeships,skillsprograms,fellowships,andcourseworkinrelevantdisciplines(seesidebar).
SuchtrainingopportunitiestargetbothscientistsandengineerswhocontributetoAIR&DinnovationsandusersofAIR&Dwhomaypossessrelevantdomainknowledge.
Inthecaseoftheformer,long-termFederalinvestmentinAIR&D,asdescribedinStrategy1,furthersupportsthegrowthofthisworkforce,boththroughtrainingthenextgenerationofresearchersandbymakingfacultypositionsmoreattractivetocurrentgraduateandpostdoctoralstudents.
Inthecaseofthelatter,newprogramsarebringingAI-relevantskillstocurrentandfutureusersofAIsystems(seesidebar).
FederalagenciesmustthereforecontinuetostrategicallyfosterexpertiseintheAIR&Dworkforcethatspansmultipledisciplinesandskillcategoriestoensuresustainednationalleadership.
118https://github.
com/touretzkyds/ai4k12/wiki119https://www.
nap.
edu/catalog/24926/assessing-and-responding-to-the-growth-of-computer-science-undergraduate-enrollments120https://www.
whitehouse.
gov/wp-content/uploads/2018/12/STEM-Education-Strategic-Plan-2018.
pdfTHENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–39–AttainingtheneededAIR&DadvancesoutlinedinthisstrategywillrequireasufficientAIR&Dworkforce.
NationswiththestrongestpresenceinAIR&Dwillestablishleadingpositionsintheautomationofthefuture.
Theywillbecomethefrontrunnersincompetencieslikealgorithmcreationanddevelopment;capabilitydemonstration;andcommercialization.
Developingtechnicalexpertisewillprovidethebasisfortheseadvancements.
WhilenoofficialAIworkforcedatacurrentlyexist,numerousrecentreportsfromthecommercialandacademicsectorsareindicatinganincreasedshortageofavailableexpertsinAI.
AIexpertsarereportedlyinshortsupply,121withdemandexpectedtocontinuetoescalate.
122High-techcompaniesarereportedlyinvestingsignificantresourcesintorecruitingfacultymembersandstudentswithAIexpertise.
123UniversitiesandindustriesarereportedlyinabattletorecruitandretainAItalent.
124AdditionalstudiesareneededtobetterunderstandthecurrentandfuturenationalworkforceneedsforAIR&D.
DataisneededtocharacterizethecurrentstateoftheAIR&Dworkforce,includingtheneedsofacademia,government,andindustry.
StudiesshouldexplorethesupplyanddemandforcesintheAIworkplace,tohelppredictfutureworkforceneeds.
AnunderstandingisneededoftheprojectedAIR&Dworkforcepipeline.
Considerationsofeducationalpathwaysandpotentialretrainingopportunitiesshouldbeincluded.
Diversityissuesshouldalsobeexplored,sincestudieshaveshownthatadiverseinformationtechnologyworkforcecanleadtoimprovedoutcomes.
125OncethecurrentandfutureAIR&Dworkforceneedsarebetterunderstood,thenappropriateplansandactionscanbeconsideredtoaddressanyexistingoranticipatedworkforcechallenges.
121"StartupsAimtoExploitaDeep-LearningSkillsGap,"MITTechnologyReview,January6,2016.
122"AItalentgrabsparksexcitementandconcern,"Nature,April26,2016.
123"ArtificialIntelligenceExpertsareinHighDemand,"TheWallStreetJournal,May1,2015.
124"Milliondollarbabies:AsSiliconValleyfightsfortalent,universitiesstruggletoholdontotheirstars,"TheEconomist,April2,2016.
125J.
W.
Moody,C.
M.
Beise,A.
B.
Woszczynski,andM.
E.
Myers,"Diversityandtheinformationtechnologyworkforce:Barriersandopportunities,"JournalofComputerInformationSystems43(2003):63-71.
THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–40–Strategy8:ExpandPublic–PrivatePartnershipstoAccelerateAdvancesinAIStrategy8isnewin2019andreflectsthegrowingimportanceofpublic-privatepartnershipsenablingAIR&D.
AmericanleadershipinscienceandengineeringresearchandinnovationisrootedintheNation'suniquegovernment-university-industryR&Decosystem.
AstheAmericanAssociationofArtsandScienceshaswritten,"America'sstandingasaninnovationleader"reliesupon"establishingamorerobustnationalGovernment-University-Industryresearchpartnership.
"126Sincethereleaseofthe2016NationalAIR&DStrategicPlan,theAdministrationhasamplifiedthisvisionofpromoting"sustainedinvestmentinAIR&Dincollaborationwithacademia,industry,internationalpartnersandallies,andothernon-FederalentitiestogeneratetechnologicalbreakthroughsinAIandrelatedtechnologiesandtorapidlytransitionthosebreakthroughsintocapabilitiesthatcontributetoU.
S.
economicandnationalsecurity.
"1Overthelastseveraldecades,fundamentalresearchininformationtechnologyconductedatuniversitieswithFederalfunding,aswellasinindustry,hasledtonew,multi-billion-dollarsectorsoftheNation'seconomy.
127Concurrentadvancesacrossgovernment,universities,andindustryhavebeenmutuallyreinforcingandhaveledtoaninnovative,vibrantAIsector.
Manyoftoday'sAIsystemshavebeenenabledbytheAmericangovernment-university-industryR&Decosystem(seesidebar).
Sincethereleaseofthe2016NationalAIR&DStrategicPlan,additionalemphasishasbeenplacedonthebenefitsofpublic-privatepartnerships.
Thesebenefitsincludestrategicallyleveragingresources,includingfacilities,datasets,andexpertise,toadvancescienceandengineeringinnovations;126RestoringtheFoundation:TheVitalRoleofResearchinPreservingtheAmericanDream(AmericanAcademyofArtsandSciences,Cambridge,MA,2014);https://www.
amacad.
org/multimedia/pdfs/publications/researchpapersmonographs/AmericanAcad_RestoringtheFoundation_Brief.
pdf.
127NationalResearchCouncilComputerScienceTelecommunicationsBoard,ContinuingInnovationinInformationTechnology(TheNationalAcademiesPress,Washington,D.
C.
,2012);https://doi.
org/10.
17226/13427.
AdvancingtheNation'sAIinnovationecosystem,spanninggovernment,universities,andindustryDeepconvolutionalneuralnetworkshaveproventobeakeyinnovationrootedinAIresearch.
AlthoughthismodelingapproachemergedfromearlyFederalinvestmentsinthelate1980s,therewerenotenoughdatanorenoughcomputationalcapabilitiesavailableatthetimeforneuralnetworkstomakeaccuratepredictions.
Throughthecombinationofariseinbigdata,today'sdata-intensivescientificmethods,andconceptualadvancesinhowtostructureandoptimizethenetworks,neuralnetworkshavere-emergedasausefulwaytoimproveaccuracyinAImodels.
Interactionsbetweenacademiaandtheprivatesector,includinggovernmentfunding,inrecentyearshavehelpedreducetheerrorrateinspeechrecognitionsystems,enablinginnovationssuchasreal-timetranslation.
126Similarly,Federalinvestmentsinreinforcementlearninginthe1980sand1990s—anapproachrootedinbehavioralpsychologythatinvolveslearningtoassociatebehaviorswithdesiredoutcomes—haveledtotoday'sdeeplearningsystems.
Throughinteractionsacrosssectors,computersareincreasinglylearninglikehumans,withoutexplicitinstruction,andreinforcementlearningisdrivingprogressinself-drivingcarsandotherformsofautomationwheremachinescanhoneskillsthroughexperience.
ReinforcementlearningwasthekeytechnologyunderlyingAlphaGo,theprogramthatdefeatedtheworld'sbestGoplayers,whichhasseenagrowingnumberofvictoriesoverprofessionalplayerssince2016.
126THENATIONALARTIFICIALINTELLIGENCER&DSTRATEGICPLAN:2019UPDATE–41–acceleratingthetransitionoftheseinnovationstopractice;andenhancingeducationandtrainingfornext-generationresearchers,technicians,andleaders.
Government-university-industryR&Dpartnershipsbringpressing,real-worldchallengesfacedbyindustrytouniversityresearchers,enabling"use-inspiredresearch";leverageindustryexpertisetoacceleratethetransitionofopenandpublishedresearchresultsintoviableproductsandservicesinthemarketplaceforeconomicgrowth;andgrowresearchandworkforcecapacitybylinkinguniversityfacultyandstudentswithindustryrepresentatives,industrysettings,andindustryjobs(seesidebar).
126,128,129,130ThesepartnershipsbuilduponjointengagementsamongFederalagenciesthatenablesynergiesinareaswhereagencies'missionsintersect.
TheNationalsobenefitsfromrelationshipsbetweenFederalagenciesandinternationalfunderswhocanworktogethertoaddresskeychallengesofmutualinterestacrossarangeofdisciplines.
Whiletherearemanystructuresandmechanismsforpublic-privatepartnerships,somecommoncategoriesforengagementinclude:1.
Individualproject-basedcollaborations.
Theseeffortsenableengagementbyuniversityresearcherswiththoseinothersectors,includingFederalagencies,industry,andinternationalorganizations,toidentifyandleveragesynergiesinareasofmutualinterest.
2.
Jointprogramstoadvanceopen,precompetitive,fundamentalresearch.
Directpartnershipsamongorganizationsacrosssectorsenablefundingandsupportforopen,precompetitive,fundamentalresearchinareasofmutualinteresttothepartners.
Ingeneral,non-FederalpartnerscontributingresearchresourcesreceivethesameintellectualpropertyrightsaffordedtotheU.
S.
GovernmentbytheBayh-DoleAct.
1313.
Collaborationstodeployandenhanceresearchinfrastructure.
CollaborationsamongFederalagencies,industry,andinternationalorganizationssignificantlyenhancethepotentialfordevelopingnewandenhancingexistingresearchinfrastructurethatinturnenablesgroundbreakingexperimentationbyresearchers.
Partnersmayofferfinancialand/orin-kindsupporttodevelopand/orenhanceresearchinfrastructure.
4.
Collaborationstoenhanceworkforcedevelopmentincludingbroadeningparticipation.
Multisectorpartnershipssetthefoundationforrigorous,engaging,andinspiringinstructionalmaterialsthatenhanceworkforcedevelopmentanddiversityinSTEMprofessions.
Ineachofthesecases,leveragingeachpartner'sstrengthsforthebenefitofallisvitallyimportanttoachievingsuccess.
128MathematicalSciencesResearchInstitutereport,"Partnerships:AWorkshoponCollaborationsbetweentheNSF/MPS&PrivateFoundations,"2015;http://library.
msri.
org/msri/Partnerships.
pdf.
129ComputingCommunityConsortium,"TheFutureofComputingResearch:Industry-AcademicCollaborations,"2016;http://cra.
org/ccc/wp-content/uploads/sites/2/2016/06/15125-CCC-Industry-Whitepaper-v4-1.
pdf.