Professor Moshe Vardi with Rice University provides an excellent article on the impacts of prioritizing efficiency over resiliency.  As we build technology and communication systems some of these key ideas should be considered as we plan for the trade-offs and between cost efficiency and the need for resilient systems needed to provide critical communication capabilities during emergency incidents.

Professor Vardi writes, “

There is a trade-off between efficiency and resilience. Efficiency requires optimal adaptation to an existing environment, while resilience is an ability to adapt to large or sudden changes in the environment. Society’s emphasis on short-term gains has long tipped the balance in favor of efficiency.

The bottom line is that resilience is a fundamental but underappreciated societal need. But both computing and economics have underemphasized resilience. In general, markets and people are quite bad at preparing for very low-probability or very long-term events.

Three recent crises – the 2021 winter storm in Texas, the COVID-19 pandemic, and the Boeing 737 Max software failure – highlight the cost of valuing efficiency over resilience and provide lessons for bringing society into balance. “

Read the rest of this thought-provoking article here:

Engineers and economists prize efficiency, but nature favors resilience – lessons from Texas, COVID-19 and the 737 Max
Moshe Y. Vardi- Professor of Computer Science, Rice University
Article in “The Conversation” May 18, 2021

https://theconversation.com/us

As the healthcare community continues to deal with the many difficulties of the COVID crisis staff burnout is becoming a major problem for many organizations.  Helping staff build their own personal resilience, or just building your own personal resilience is a key part of making sure your organization is prepared for all the other incidents and issues we face every day.

Building Your Resilience” published by The American Psychological Association provides great advice on how to build personal psychological resilience.

Their article provides a road map for personal resilience to help not just with dealing with COVID but for allowing you to deal with everyday stresses.  As the article states, “There are many aspects of your life you can control, modify, and grow with. That’s the role of resilience. Becoming more resilient not only helps you get through difficult circumstances, it also empowers you to grow and even improve your life along the way.”

Article URL https://www.apa.org/topics/resilience/#

 

This spring, the DHS Science and Technology Directorate (S&T) field tested FirstNet’s Push-to-Talk (FNPTT) application, which is a standards-based, mission-critical push-to-talk (MCPTT) app. The app allows voice, video, and data communications with high priority and low delays with the press of a button.

Over the course of one month, 50 federal government organizations tested the app in different scenarios that reflected real-world responses. MCPTT is the public safety mission-critical standard set by the 3rd Generation Partnership Project (3GPP), which is responsible for LTE and 5G global standards. Representatives from the Joint Wireless Program Management Office (JWPMO) and FirstNet AT&T were also involved with the field test.

Many of the available apps in the marketplace are not standards-compliant, which poses a significant challenge for users of current push-to-talk solutions. It presents a barrier for interoperability and sustainability, with different proprietary solutions and associated costs.  MCPTT includes key characteristics useful for public safety: high availability, reliability, and low latency; 1:1 calls and group calls; emergency calling; device-to-device direct communication and location reporting.

For more information see the full article at the Department of Homeland Security Science and Technology website.  https://www.dhs.gov/science-and-technology/news/2021/09/28/feature-article-st-tests-firstnet-push-talk-app-real-world-scenarios

“On a November evening five years ago, Tennessee firefighters fielded calls about a fire burning in Great Smoky Mountains National Park, near Gatlinburg. A few days later, high winds pushed the fire into the park’s Chimneys Picnic Area—out of control. 

Today, the Chimney Tops 2 Fire is considered one of the most devastating fires in the state’s history. Cell phone reception went down. The region’s 9-1-1 system couldn’t handle the call load. It caused an estimated $2 billion in damage—more than 17,000 acres were burned and 14 people died.

At the time of the fire the county ‘s emergency alert system had only 1% of the population enrolled to receive emergency alerts.  While the system was in place to send the alerts the work to enroll the local community had not been completed which made the system ineffective when needed for this emergency. 

A critical part of preparing our community and our local healthcare organizations for emergencies is making sure the available communication tools are used by our staff to receive critical communications.   PUSHECS has created a page with links to many of our regions City, County and Federal alert systems to assist your efforts to communicate with your staff the many alert system in place that they can sign up for so they can receive effective and timely emergency alert notifications.  Consider making this information available on your staff emergency preparedness programs.

 

See the PUSHECS page on regional and national alert systems for more information

A black swan event has, unfortunately, nothing to do with Natalie Portman or Western Australia; it describes an event that is extremely unlikely, but can cause massive upheaval. For example, the 2008 global recession or, say, pretty much all of 2020.

By definition, no one sees a black swan event coming — but researchers at Stanford are attempting to change that. They are building a computational method to try and predict when the next graceful neck will raise its head.

“This work is exciting because it’s a chance to take the knowledge and the computational tools that we’re building in the lab and use those to better understand — even predict or forecast — what happens in the world surrounding us,” Bo Wang, assistant professor of bioengineering at Stanford and senior author of the study, told Stanford News…….

Read the complete text of this article at BigThink  by author B. David Zarley

https://bigthink.com/the-present/predict-unpredictable-black-swan-events/

Excuse me, what just happened? Resilience is tough when your failure is due to a ‘sequence of events that was almost impossible to foresee’

 

Thinking about how to design and create resilient systems is tough, especially when the failure path is not apparent or is buried deep within the system design.  So called “Black Swan” events are those once in a life time catastrophic events whose failure paths often only become apparent after the event but expose flaws which in hindsight were all to apparent.  Read this great article from the “The Register” website on some basic steps to help design and test your critical communication and data systems to help avoid that “Black Swan” event.  https://www.theregister.com/2021/06/12/how_to_make_tech_systems_resilient/

 

For years, people had warned that New Orleans was vulnerable – but when a hurricane came close to destroying the city, the reaction was muted. Some people took the near miss as a warning – others, as confirmation that there was nothing to worry about.

So why do we struggle to prepare for disasters? And why don’t we draw the obvious lessons from clear warnings?

In the podcast “That Turn to Pascagoula” written and presented by Tim Harford with Andrew Wright, the authors talk about how we do not see and do not act to prevent foreseeable disasters.  A thought provoking podcast for anyone interested in how we should be working to mitigate the impacts of disasters and failures before they arrive.

Additional information and readings on the subject can be found on Tim’s website.  https://timharford.com/2020/07/cautionary-tales-that-turn-to-pascagoula/

Additional suggested readings on the subject. 

Read about Hurricane Ivan in The Ostrich Paradox by Howard Kunreuther and Robert Meyer, and other forebodings of disaster are recounted in Predictable Surprises by Max Bazerman and Michael Watkins. 

 

Verizon announced another delay for its planned January 2021 3G cellular network shutdown, but time is probably running out for 3G. All the major cellular carriers are planning to shut down their 3G networks in the near future. With these impending shutdowns, your organization should have a plan in place to validate if wireless devices are still using 3G networks and be actively implementing a migration plan for any soon to be obsolete solutions.

Bill Menezes, director analyst at tech research firm Gartner said “Verizon’s decision to delay its 3G shutdown probably has less to do with mobile phone customers still using the network — which are likely few in number — and more to do with “internet of things” devices, such as smart utility meters and home burglar alarms that are still connected to 3G”. For the corporate market, SCADA, IoT, transportation tracking, and networking devices are the prime candidates for systems still using 3G technology.  Network backup solutions, remote monitoring, and telematics solutions should all be reviewed to see if they are using 3G cellular connections. For network and communication managers special consideration may be needed for edge routers using 3G for WAN backup or out-of-band remote access as the obsolete 3G connection may not be apparent until a primary connection outage. Think about those CradlePoint routers you deployed to a branch office a few years ago with a 3G SIM card.

During a January 5th interview with the website “LightReading” Verizon spokesperson Kevin King was quoted as saying “our 3G network is operational, and we don’t have a plan to shut it down at this time.” This signals another delay in Verizon’s announced 3G shutdown plans, which had previously been announced for the end of 2019 and again for 2020. Verizon and other carriers stopped activating 3G devices in 2018 but many 3G devices are still operational in the field. In 2021, 3G mobile connections are expected to shrink to 5.7% of total cellular connections, but that still is more than 25 million devices in the US. This latest delay is only a reprieve for anyone still using 3G devices as the spokesperson clarified that Verizon is still planning to shutter the 3G network but has not announced the new date.

 It is expected that the carriers will sunset their 3G networks rather than implementing complete shutdowns on a given date. A sunset strategy may see the carrier’s 3G network equipment decommissioned as it fails or cell towers are upgraded. The sunset strategy may help extend the time to move services to 4G networks but also means that many organizations may not notice their 3G devices are slowly failing and failure will be dependent on local 3G network availability. A failure of IoT, SCADA, and other devices using 3G networking could cause some nasty surprises for organizations and lead to emergency or unplanned equipment replacement and upgrades. Now is the time to prepare so you can identify what needs to be upgraded, get your budgets approved, and start the process of making the required changes and equipment updates.

AT&T’s website provides a February 2022 date for their planned 3G network shutdown https://www.att.com/support/article/wireless/KM1324171, while T-Mobile hints that they plan to sunset their 3G network but have provided no official announcement. T-Mobile’s website currently provides no information and no planned shutdown date.

 

Sources

Verizon indefinitely delays 3G network shutdown”

By Mike Dano, LightReading, Jan 5, 2021

 

When Will 3G Be Retired? Here are the Timelines for Verizon, T-Mobile & AT&T”

By Jess Barnes, Cord Cutters News, Aug 21, 2020

 

Verizon delaying shutdown of its 3G wireless network”

By Clare Duffy, CNN Business, January 7, 2021

 

3G Networks are Becoming Extinct

Conklin, A., Yahoo Finance,  March 26, 2020

 

Verizon Support Website

https://www.verizon.com/support/knowledge-base-218813/

AT&T Support Website

https://www.att.com/support/article/wireless/KM1324171

SpaceX donated seven Starlink satellite terminals to Washington State’s Emergency Management Department (EMD) to provide emergency data services as part of the State’s 2020 wildfire response.  Some of the terminals were deployed to provide internet connectivity for the town of Malden, where it is estimated that 80% of the homes were destroyed in this small Washington town.  The Starlink system was quickly set up to provide emergency communications and broadband services for local residents. 

Malden_Starlink_Data_Svc_2020_Wildfire
Image credits WA EMD

Steven Friederich, with EMD explained, “the fire come through town and it burned a good chunk of the area, including the fire station and the post office. There simply hasn’t been a way to get a fast and reliable Internet connection there for the public to use…This is a device we could definitely utilize should we have more wildfires or even larger disasters, such as a Cascadia Subduction earthquake event, where communication problems would be a huge hurdle”.

The terminals are also being used for incident command at the Bonney Lake wildfire.  Richard Hall with Washington State’s Military Department IT was quoted as saying “I have never set up any tactical satellite equipment that has been as quick to set up, and anywhere near as reliable.”

The Starlink terminals are part of a new satellite network service currently being deployed by SpaceX.  The new satellite-based broadband service is promising low latency and high bandwidth.  While the system is still in development SpaceX has published testing results showing a 30-millisecond latency and bandwidth of up to 60 Mbps.  The company is preparing to launch a public beta for Starlink later this year for residents in the northern US and Canada.

As already demonstrated the Starlink system may provide Emergency Management and Healthcare IT/Communications access to new satellite-based data services for use in emergency situations and as a backup for traditional land-based systems. New satellite-based data networks will be an interesting service to keep an eye on in the next few years to see if these new systems can provide critical data network connectivity even if land-based systems fail or are damaged in emergency situations.

 

Sources

SpaceX Is Providing Satellite Internet Service to Towns Hit by Wildfires

By Michael Kan   September 29, 2020

https://www.pcmag.com/news/spacex-is-providing-satellite-internet-service-to-towns-hit-by-wildfires

 

 

SpaceX’s Satellite Internet Plans for Mid

-2020 Launch in the USBy Michael Kan  October 23, 2019

https://www.pcmag.com/news/spacexs-satellite-internet-plans-for-mid-2020-launch-in-the-us

 

Starlink puts towns devastated by wildfires online for disaster relief workers

Devin Coldewey@techcrunch September 29, 2020

Starlink puts towns devastated by wildfires online for disaster relief workers

The Internet experienced another large-scale outage on July 17th, 2020 that prevented access to many major websites and Internet-based services for about 25 minutes. The outage was the result of a problem with Internet DNS services provided by Cloudflare, impacting Internet services in many parts of the US, including the Seattle area.

Cloudflare, a company that is not a household name, provides world-wide Internet services such as DNS services, Content Delivery Networks, and Denial of Service protection for over 500,000 companies, many in the US and in the Hospital and Health Care industry. Cloudflare stressed that this was not a hacker attack, but a simple configuration error.

The outage follows similar outages over the last two years involving Cloudflare, Verizon, and T-Mobile highlighting the highly interdependent nature of today’s Internet.

Cloudflare’s company blog page provided a brief technical overview of the outage.

“… while working on an unrelated issue with a segment of the backbone from Newark to Chicago, our network engineering team updated the configuration on a router… This configuration contained an error that caused all traffic across our backbone to be sent to Atlanta. This quickly overwhelmed the Atlanta router and caused Cloudflare network locations connected to the backbone to fail.” This simple error by a single engineering team impacted 50% of the traffic on their network.

These outages provide an insight into issues that everyone in the Health Care IT and Communications communities should consider when planning the resiliency of their critical or essential services. Today’s Internet provides critical links in our IT and communications infrastructures which we may not even consider until the service is unavailable. A simple example; internal pager services often use an email gateway interface to your paging vendor often sends this traffic over the Internet. As these nationwide outages demonstrate it is critical that we are aware of the critical links in the chain that makes up these services and plan alternate means of providing the service during a failure event. See the PUSHECS “Ideas and Concepts” page for more information on PACE planning and Resiliency Planning.