{"id":11447,"date":"2019-01-16T17:52:29","date_gmt":"2019-01-16T17:52:29","guid":{"rendered":"https:\/\/blog.nearfuturelaboratory.com\/?p=11447"},"modified":"2019-10-23T09:30:28","modified_gmt":"2019-10-23T09:30:28","slug":"when-automation-bites-back","status":"publish","type":"post","link":"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/","title":{"rendered":"When Automation Bites Back"},"content":{"rendered":"\n<p><strong>The business of dishonest automation and how the engineers, data scientists and designers behind it can fix it<\/strong><\/p>\n\n\n\n<p>&#8220;<em>The pilots fought continuously until the end of the flight<\/em>&#8220;, said Capt. Nurcahyo Utomo, the head of the investigation of Lion Air Flight 610 that crashed on October 29, 2018, killing the 189 people aboard. The analysis of the black boxes had revealed that the Boeing 737&#8217;s nose was repeatedly forced down, apparently by an automatic system receiving incorrect sensor readings. During 10 minutes preceding the tragedy, the pilots tried 24 times to manually pull up the nose of the plane. They struggled against a malfunctioning anti-stall system that they did not know how to disengage for that specific version of the plane.<\/p>\n\n\n\n<p>That type of dramatic scene of humans struggling with a stubborn automated system belongs to pop culture. In the famous scene of the 1968 science-fiction film \u201c2001: A Space Odyssey&#8221;, the astronaut Dave asks HAL (Heuristically programmed ALgorithmic computer) to open a pod bay door on the spacecraft, to which HAL responds repeatedly, \u201c<em>I\u2019m sorry, Dave, I\u2019m afraid I can\u2019t do that<\/em>&#8220;.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"HAL 9000: &quot;I&#039;m sorry Dave, I&#039;m afraid I can&#039;t do that&quot;\" width=\"660\" height=\"371\" src=\"https:\/\/www.youtube.com\/embed\/ARJ8cAGm6JE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<h2><strong>1. The commodification of automation<\/strong><\/h2>\n\n\n\n<p>Thankfully, the contemporary applications of digital automation are partial and do not take the shape of an &#8220;artificial general intelligence&#8221; like HAL. However, the computational tasks that once were exclusively applied to automate human jobs in critical environments like a cockpit have reached people&#8217;s everyday lives (e.g. automated way-finding, smart thermostat) and the techniques often deployed for more frivolous but yet very lucrative objectives (e.g. targeted advertisements, prioritizing the next video to watch on YouTube).<\/p>\n\n\n\n<blockquote class=\"wp-block-quote\"><p>\u201cWhat concerns me is that many engineers, data scientists, designers and decision-makers bring digital frictions into people\u2019s everyday life because they do not employ approaches to foresee the limits and implications of their work\u201d<br><\/p><\/blockquote>\n\n\n\n<p>The automated systems that once relied on programmed instructions based on their author&#8217;s understanding of the world now also model their behavior from the patterns found in datasets of sensors and human activities. As the application of these Machine Learning techniques becomes widespread, digital automation is becoming a commodity with systems that perform at Internet scale one task with no deep understanding of human context. These systems are trained to complete that &#8220;one&#8221; job, but there are evidences that their behavior, like HAL or a Boeing 737 anti-stall system, can turn against their user\u2019s intentions when things do not go as expected.<\/p>\n\n\n\n<h2>2.  The clumsy edges<\/h2>\n\n\n\n<p>Recent visual ethnographies at Near Future Laboratory like &nbsp;<a href=\"http:\/\/tuxsax.nearfuturelaboratory.com\/\">#TUXSAX<\/a> and <a href=\"http:\/\/curiousrituals.nearfuturelaboratory.com\/\">Curious Rituals<\/a> uncovered&nbsp;some implications of that commodification of automation. In a completely different scale of dramatic consequences that brought down Lion Air Flight 610, these observations highlight how some digital solutions leave people with a feeling of being &#8220;locked in&#8221; with no &#8220;escape&#8221; key to disengage from a stubborn behavior. A wide majority of these digital frictions provoke harmless micro-frustrations in people&#8217;s everyday lives. They manifest themselves through poorly calibrated systems and a design that disregards edge cases. For instance, it is common to experience a voice assistant unable to understand a certain accent or pronunciation or a navigation system that misleads a driver due to location inaccuracies, obsolete road data or incorrect traffic information.<\/p>\n\n\n\n<figure class=\"wp-block-embed-vimeo wp-block-embed is-type-video is-provider-vimeo wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"Curious Rituals: Calling Gerardo\" src=\"https:\/\/player.vimeo.com\/video\/368229317?dnt=1&amp;app_id=122963\" width=\"660\" height=\"371\" frameborder=\"0\" allow=\"autoplay; fullscreen\" allowfullscreen><\/iframe>\n<\/div><figcaption>Curious rituals is a fiction that showcases the gaps and junctures that glossy corporate videos on the &#8220;future of technology&#8221; do not reveal. Source: <a href=\"http:\/\/curiousrituals.nearfuturelaboratory.com\/\u00a8\">Curious Rituals<\/a>.<\/figcaption><\/figure>\n\n\n\n<p>These clumsy automations can be mitigated but will not disappear because it became impossible to design contingency plans for all unexpected limitations or consequences. However, other types of stubborn autonomous behaviours are intentionally designed as the core of business models that trades human control for convenience.<\/p>\n\n\n\n<h2>3. The business of dishonest automation<\/h2>\n\n\n\n<p>Many techniques to automate everyday tasks allow organizations to reduce costs and increase revenues. Some members of the tech industry employ these new technological capabilities to lock customers or workers into behaviors for which they have no legitimate need or desire. Those systems are typically designed to resist from their user&#8217;s demands AND are hard to disengage. Let me give you a couple of examples of what I call \u201cdishonest automations\u201d:<\/p>\n\n\n\n<p><strong>3.1. Data obesity<\/strong><\/p>\n\n\n\n<p>Automatic cloud backup systems have become a default feature of operating systems. They externalize the storage of personal photos, emails, contacts and other bits of digital life. Their business model encourages customers to endlessly accumulate more content without a clear alternative that promotes a proper hygiene with their data (i.e. nobody has yet come up with \u201cMarie Kondo for Dropbox &#x2122;\u201d). Regardless of the promises of the providers, it becomes harder for people to declutter their digital lives from a cloud storage service. <\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" width=\"1024\" height=\"768\" src=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-7-2-1024x768.jpg\" alt=\"\" class=\"wp-image-11453\" srcset=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-7-2-1024x768.jpg 1024w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-7-2-300x225.jpg 300w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-7-2-768x576.jpg 768w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-7-2.jpg 2048w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption>Upgrade your storage to continue backing up: an automatic cloud backup system that locks in its user, leaving no alternative to the accumulation of content.<\/figcaption><\/figure>\n\n\n\n<p><strong>3.2. Systemic obsolescence<\/strong><\/p>\n\n\n\n<p>Today&#8217;s apps automatic updates often increase the demand of resources and processing power for cosmetic improvements almost in a deliberate attempt to make a hardware obsolete and the software harder to operate. After years of impunity, there is now a bigger conscience against systemic obsolescence because it is wasteful and exploits customers.<br><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" width=\"1024\" height=\"155\" src=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-25-1024x155.jpg\" alt=\"\" class=\"wp-image-11454\" srcset=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-25.jpg 1024w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-25-300x45.jpg 300w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-25-768x116.jpg 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><strong>3.3. Digital attention<\/strong><\/p>\n\n\n\n<p>As content grows exponentially on the Internet, (social) media companies rely increasingly on automation to filter and direct information to each one of their users. For instance, YouTube automates billions of videos to play next for 1.5 billion users. These algorithms aim at promoting content for higher engagement and tend to guide people against their interest. <br><\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p>In the light of these examples of clumsy and dishonest automation, what concerns me is that many engineers, data scientists, designers and decision-makers bring these frictions into people\u2019s everyday life because they do not employ approaches to foresee the limits and implications of their work. Apart from the engineering of efficient solutions, automation requires professionals to think about the foundations and consequences of their practice that transcend any Key Performance Indicator of their organization.<\/p>\n\n\n\n<h2>4. The design for humane automation<\/h2>\n\n\n\n<p>The design of automation is not about removing the presence of humans. It is about the design of humane, respectful and trustful systems that automate some aspects of human activities. When <a href=\"https:\/\/towardsdatascience.com\/the-mindset-for-innovation-with-data-science-fc51605a4867\">working with data scientists, designers and engineers<\/a> in that domain, we envision systems beyond the scope of the \u201cuser\u201d and the \u201ctask\u201d to automate. I encourage teams to a) learn from the past b) critique the present and c) debate the future. Let me explain:<\/p>\n\n\n\n<p><strong>4.1. Learn&nbsp;from&nbsp;the&nbsp;past<\/strong><\/p>\n\n\n\n<p>When it comes to automation, the acquisition of knowledge in academia and in the industry are not separate pursuits. Over the last 50 years, there has been an extensive body of work produced in research institutions on the implications of automating manual tasks and decision-making. The key findings have helped save money in critical environments and prevent numerous deadly errors (e.g. in cockpits).<\/p>\n\n\n\n<p>Today, that knowledge is not translated into everyday tasks. For instance, many engineers or data scientists do not master concepts like automation bias (i.e. the propensity for humans to favor suggestions from automated decision-making systems) or automation complacency (i.e. decreased human attention to monitor automated results) theorized by research communities in Science and Technology Studies or Human-Computer Interaction. Sadly, only a few organizations promote platforms that gather academics, artists, engineers, data scientists and designers. Industries in the process of digitization would greatly profit from this type cross-pollination of professionals who learn from considerations that already emerged outside of their discipline.<\/p>\n\n\n\n<p><strong>4.2. Critique&nbsp;the&nbsp;present<\/strong><\/p>\n\n\n\n<p>I believe that the professionals involved in the business of automating human activities should be persistent critical reviewers of the solutions deployed by their peers. They should become stalkers of how people deal today with the clumsy, the dishonest, the annoying, the absurd and any other awkward emerges of digital technologies in their modern lives.<\/p>\n\n\n\n<figure class=\"wp-block-embed-vimeo wp-block-embed is-type-video is-provider-vimeo wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"#TUXSAX\" src=\"https:\/\/player.vimeo.com\/video\/184133960?dnt=1&amp;app_id=122963\" width=\"660\" height=\"371\" frameborder=\"0\" allow=\"autoplay; fullscreen\" allowfullscreen><\/iframe>\n<\/div><figcaption>#TUXSAX is an invitation to engage with these knotty, gnarled edges of technology. It provides some raw food for thoughts to consider the mundane frictions between people and technologies. Do we want to mitigate, or even eliminate these frictions? Source: <a href=\"https:\/\/medium.com\/@girardin\/documenting-the-state-of-contemporary-technology-ddf2cc6c8de4\">Documenting the State of Contemporary Technology<\/a>.<\/figcaption><\/figure>\n\n\n\n<p>When properly documented, these observations offer a complementary form of inspiration to the multitude of \u201cnaive optimism\u201d and glamorous utopian visions of the tech industry. They provide material for professionals to question arguably biased goals of automation. Moreover, they set the stage to define attainable objectives in their organization (e.g. what does smart\/intelligent mean?, how to measure efficiency?, what must become legible?).<\/p>\n\n\n\n<p><strong>4.3. Debate&nbsp;the&nbsp;future<\/strong><\/p>\n\n\n\n<p>In today&#8217;s Internet, the design of even the most simple application or connected object has become a complex endeavour. They are built on balkanized Operating Systems, stacks of numerous protocols, versions, frameworks, and other packages of reusable code. The mitigation of digital frictions goes beyond the scope of a &#8220;Quality Assurance&#8221; team that guarantees the sanity of an application. They are also about documenting implications on the context the technologies live, unintended consequences and &#8216;what if&#8217; scenarios.<br><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" width=\"800\" height=\"533\" src=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-5.jpeg\" alt=\"\" class=\"wp-image-11455\" srcset=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-5.jpeg 800w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-5-300x200.jpeg 300w, https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-5-768x512.jpeg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><figcaption>It&#8217;s easy to get all Silicon Valley when drooling over the possibility of a world chock-full of self-driving cars. However, when an idea moves from speculation to designed product it is necessary to consider the many facets of its existence\u200a-\u200athe who, what, how, when, why of the self-driving car. To address these questions, we took a sideways glance at it by forcing ourselves to write the quick-start guide for a typical self-driving car. Source: <a href=\"https:\/\/medium.com\/design-fictions\/the-world-of-self-driving-cars-b1f7ade18931\">The World of Self-Driving Cars<\/a>.<\/figcaption><\/figure>\n\n\n\n<p>Typically, Design Fiction is an approach to spark a conversation and anticipate the larger questions regarding the automation of human activities. For instance, we produced <a href=\"http:\/\/qsg.nearfuturelaboratory.com\/\">Quick Start Guide of Amazon Helios: Pilot<\/a>, a fictional autonomous vehicle. In that project, we identified the key systems that implicate the human aspects of a self-driving car and we brought to life such experiences in a very tangible, compelling fashion for designers, engineers, and anyone else involved in the development of automated systems. Through its collective production, the Quick Start Guide became a totem through which anybody could discuss the consequences, raise design considerations and shape decision-making.<\/p>\n\n\n\n<h2>5. The business of trust<\/h2>\n\n\n\n<p>Like many technological evolution, the automation of everyday life does not come without the frictions of trading control for convenience. However, the consequences are bigger than mitigating edge cases. They reflect human, organization or society choices. The choice of deploying systems that mislead about their intentions in conflict with people and society&#8217;s interests.<\/p>\n\n\n\n<p>In his seminal work on Ubiquitous Computing in the 90s, Mark Weiser strongly influenced the current \u201cthird wave\u201d in computing, when technology recedes into the background of people&#8217;s lives. Many professionals in the tech industry (including me) embraced his description of Calm technology that &#8220;<em>informs but doesn&#8217;t demand our focus or attention<\/em>.&#8221; However, what Weiser and many others (including me) did not anticipate is an industry of dishonest automation or solutions that turn against their user\u2019s intentions when things do not go as planned. Nor did we truly anticipate the scale in which automation can bite back the organizations that deploy them with backslashes from their customers, society as well as policymakers.<\/p>\n\n\n\n<figure class=\"wp-block-embed-instagram wp-block-embed is-type-rich is-provider-instagram\"><div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"instagram-media\" data-instgrm-captioned data-instgrm-permalink=\"https:\/\/www.instagram.com\/p\/BSyo6bMlvfa\/?utm_source=ig_embed&amp;utm_campaign=loading\" data-instgrm-version=\"12\" style=\" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:658px; min-width:326px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);\"><div style=\"padding:16px;\"> <a href=\"https:\/\/www.instagram.com\/p\/BSyo6bMlvfa\/?utm_source=ig_embed&amp;utm_campaign=loading\" style=\" background:#FFFFFF; line-height:0; padding:0 0; text-align:center; text-decoration:none; width:100%;\" target=\"_blank\"> <div style=\" display: flex; flex-direction: row; align-items: center;\"> <div style=\"background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;\"><\/div> <div style=\"display: flex; flex-direction: column; flex-grow: 1; justify-content: center;\"> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;\"><\/div> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;\"><\/div><\/div><\/div><div style=\"padding: 19% 0;\"><\/div> <div style=\"display:block; height:50px; margin:0 auto 12px; width:50px;\"><svg width=\"50px\" height=\"50px\" viewBox=\"0 0 60 60\" version=\"1.1\" xmlns=\"https:\/\/www.w3.org\/2000\/svg\" xmlns:xlink=\"https:\/\/www.w3.org\/1999\/xlink\"><g stroke=\"none\" stroke-width=\"1\" fill=\"none\" fill-rule=\"evenodd\"><g transform=\"translate(-511.000000, -20.000000)\" fill=\"#000000\"><g><path d=\"M556.869,30.41 C554.814,30.41 553.148,32.076 553.148,34.131 C553.148,36.186 554.814,37.852 556.869,37.852 C558.924,37.852 560.59,36.186 560.59,34.131 C560.59,32.076 558.924,30.41 556.869,30.41 M541,60.657 C535.114,60.657 530.342,55.887 530.342,50 C530.342,44.114 535.114,39.342 541,39.342 C546.887,39.342 551.658,44.114 551.658,50 C551.658,55.887 546.887,60.657 541,60.657 M541,33.886 C532.1,33.886 524.886,41.1 524.886,50 C524.886,58.899 532.1,66.113 541,66.113 C549.9,66.113 557.115,58.899 557.115,50 C557.115,41.1 549.9,33.886 541,33.886 M565.378,62.101 C565.244,65.022 564.756,66.606 564.346,67.663 C563.803,69.06 563.154,70.057 562.106,71.106 C561.058,72.155 560.06,72.803 558.662,73.347 C557.607,73.757 556.021,74.244 553.102,74.378 C549.944,74.521 548.997,74.552 541,74.552 C533.003,74.552 532.056,74.521 528.898,74.378 C525.979,74.244 524.393,73.757 523.338,73.347 C521.94,72.803 520.942,72.155 519.894,71.106 C518.846,70.057 518.197,69.06 517.654,67.663 C517.244,66.606 516.755,65.022 516.623,62.101 C516.479,58.943 516.448,57.996 516.448,50 C516.448,42.003 516.479,41.056 516.623,37.899 C516.755,34.978 517.244,33.391 517.654,32.338 C518.197,30.938 518.846,29.942 519.894,28.894 C520.942,27.846 521.94,27.196 523.338,26.654 C524.393,26.244 525.979,25.756 528.898,25.623 C532.057,25.479 533.004,25.448 541,25.448 C548.997,25.448 549.943,25.479 553.102,25.623 C556.021,25.756 557.607,26.244 558.662,26.654 C560.06,27.196 561.058,27.846 562.106,28.894 C563.154,29.942 563.803,30.938 564.346,32.338 C564.756,33.391 565.244,34.978 565.378,37.899 C565.522,41.056 565.552,42.003 565.552,50 C565.552,57.996 565.522,58.943 565.378,62.101 M570.82,37.631 C570.674,34.438 570.167,32.258 569.425,30.349 C568.659,28.377 567.633,26.702 565.965,25.035 C564.297,23.368 562.623,22.342 560.652,21.575 C558.743,20.834 556.562,20.326 553.369,20.18 C550.169,20.033 549.148,20 541,20 C532.853,20 531.831,20.033 528.631,20.18 C525.438,20.326 523.257,20.834 521.349,21.575 C519.376,22.342 517.703,23.368 516.035,25.035 C514.368,26.702 513.342,28.377 512.574,30.349 C511.834,32.258 511.326,34.438 511.181,37.631 C511.035,40.831 511,41.851 511,50 C511,58.147 511.035,59.17 511.181,62.369 C511.326,65.562 511.834,67.743 512.574,69.651 C513.342,71.625 514.368,73.296 516.035,74.965 C517.703,76.634 519.376,77.658 521.349,78.425 C523.257,79.167 525.438,79.673 528.631,79.82 C531.831,79.965 532.853,80.001 541,80.001 C549.148,80.001 550.169,79.965 553.369,79.82 C556.562,79.673 558.743,79.167 560.652,78.425 C562.623,77.658 564.297,76.634 565.965,74.965 C567.633,73.296 568.659,71.625 569.425,69.651 C570.167,67.743 570.674,65.562 570.82,62.369 C570.966,59.17 571,58.147 571,50 C571,41.851 570.966,40.831 570.82,37.631\"><\/path><\/g><\/g><\/g><\/svg><\/div><div style=\"padding-top: 8px;\"> <div style=\" color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;\"> View this post on Instagram<\/div><\/div><div style=\"padding: 12.5% 0;\"><\/div> <div style=\"display: flex; flex-direction: row; margin-bottom: 14px; align-items: center;\"><div> <div style=\"background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(0px) translateY(7px);\"><\/div> <div style=\"background-color: #F4F4F4; height: 12.5px; transform: rotate(-45deg) translateX(3px) translateY(1px); width: 12.5px; flex-grow: 0; margin-right: 14px; margin-left: 2px;\"><\/div> <div style=\"background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(9px) translateY(-18px);\"><\/div><\/div><div style=\"margin-left: 8px;\"> <div style=\" background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;\"><\/div> <div style=\" width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)\"><\/div><\/div><div style=\"margin-left: auto;\"> <div style=\" width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);\"><\/div> <div style=\" background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);\"><\/div> <div style=\" width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);\"><\/div><\/div><\/div><\/a> <p style=\" margin:8px 0 0 0; padding:0 4px;\"> <a href=\"https:\/\/www.instagram.com\/p\/BSyo6bMlvfa\/?utm_source=ig_embed&amp;utm_campaign=loading\" style=\" color:#000; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none; word-wrap:break-word;\" target=\"_blank\">#curiousrituals #classic #vendingmachine<\/a><\/p> <p style=\" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;\">A post shared by <a href=\"https:\/\/www.instagram.com\/nicolasnova\/?utm_source=ig_embed&amp;utm_campaign=loading\" style=\" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px;\" target=\"_blank\"> nicolas nova<\/a> (@nicolasnova) on <time style=\" font-family:Arial,sans-serif; font-size:14px; line-height:17px;\" datetime=\"2017-04-12T16:00:52+00:00\">Apr 12, 2017 at 9:00am PDT<\/time><\/p><\/div><\/blockquote><script async src=\"\/\/www.instagram.com\/embed.js\"><\/script>\n<\/div><\/figure>\n\n\n\n<p>These implications suggest an alternative paradigm that transcend the purely technological and commercial for any organization involved in the business of digital automation. For instance, a paradigm that promotes respectful (over efficient), legible (over calm) and honest (over smart) technologies. Those are the types of values that emerge when professionals (e.g. engineers, data scientists, designers, decision-makers, executives) wander outside their practice, apply critical thinking to uncover dishonest behaviors, and use fictions to take decisions that consider implications beyond the scope of the &#8220;user&#8221; and the &#8220;task&#8221; to automate.<br><\/p>\n\n\n\n<p>I believe that the organizations in the business of automation that maintain the status-quo and do not evolve into a business of trust might eventually need to deal with a corroded reputation and its effects on their internal values, the moral of employees, the revenues and ultimately the stakeholders trust.<br><\/p>\n\n\n\n<p><br><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The business of dishonest automation and how the engineers, data scientists and designers behind it can fix it &#8220;The pilots fought continuously until the end of the flight&#8220;, said Capt. Nurcahyo Utomo, the head of the investigation of Lion Air Flight 610 that crashed on October 29, 2018, killing the 189 people aboard. The analysis &hellip; <a href=\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">When Automation Bites Back<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":11448,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[50,53,65,74,79,1168],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v14.9 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>When Automation Bites Back - Near Future Laboratory<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"When Automation Bites Back - Near Future Laboratory\" \/>\n<meta property=\"og:description\" content=\"The business of dishonest automation and how the engineers, data scientists and designers behind it can fix it &#8220;The pilots fought continuously until the end of the flight&#8220;, said Capt. Nurcahyo Utomo, the head of the investigation of Lion Air Flight 610 that crashed on October 29, 2018, killing the 189 people aboard. The analysis &hellip; Continue reading When Automation Bites Back\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/\" \/>\n<meta property=\"og:site_name\" content=\"Near Future Laboratory\" \/>\n<meta property=\"article:published_time\" content=\"2019-01-16T17:52:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-10-23T09:30:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-28.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@nearfuturelab\" \/>\n<meta name=\"twitter:site\" content=\"@nearfuturelab\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#organization\",\"name\":\"Near Future Laboratory\",\"url\":\"https:\/\/blog.nearfuturelaboratory.com\/\",\"sameAs\":[\"https:\/\/www.instagram.com\/nearfuturelaboratory\/\",\"https:\/\/www.linkedin.com\/company\/near-future-laboratory\/\",\"https:\/\/twitter.com\/nearfuturelab\"],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/10\/NearFutureLaboratoryLogo-CS4.jpg\",\"width\":1049,\"height\":206,\"caption\":\"Near Future Laboratory\"},\"image\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#website\",\"url\":\"https:\/\/blog.nearfuturelaboratory.com\/\",\"name\":\"Near Future Laboratory\",\"description\":\"Clarify Today, Design Tomorrow\",\"publisher\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/blog.nearfuturelaboratory.com\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/blog.nearfuturelaboratory.com\/wp-content\/uploads\/2019\/01\/medium-passive-aggresive-computing-28.jpg\",\"width\":1024,\"height\":600},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#webpage\",\"url\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/\",\"name\":\"When Automation Bites Back - Near Future Laboratory\",\"isPartOf\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#primaryimage\"},\"datePublished\":\"2019-01-16T17:52:29+00:00\",\"dateModified\":\"2019-10-23T09:30:28+00:00\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/\"]}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#webpage\"},\"author\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#\/schema\/person\/d6a8715efad844f2958d47460d4b9ef0\"},\"headline\":\"When Automation Bites Back\",\"datePublished\":\"2019-01-16T17:52:29+00:00\",\"dateModified\":\"2019-10-23T09:30:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#webpage\"},\"commentCount\":2,\"publisher\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/2019\/01\/16\/when-automation-bites-back\/#primaryimage\"},\"articleSection\":\"Design Fiction,Design Technology,Failure,glitch,HCI,self-driving car\",\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#\/schema\/person\/d6a8715efad844f2958d47460d4b9ef0\",\"name\":\"Fabien\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/blog.nearfuturelaboratory.com\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"http:\/\/0.gravatar.com\/avatar\/c7b0c2bfac1d6e26ca5286c45530ffaf?s=96&d=mm&r=g\",\"caption\":\"Fabien\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/posts\/11447"}],"collection":[{"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/comments?post=11447"}],"version-history":[{"count":10,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/posts\/11447\/revisions"}],"predecessor-version":[{"id":11515,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/posts\/11447\/revisions\/11515"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/media\/11448"}],"wp:attachment":[{"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/media?parent=11447"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/categories?post=11447"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.nearfuturelaboratory.com\/wp-json\/wp\/v2\/tags?post=11447"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}