Commissioned Internal Review Finds @StateDept’s Consular Consolidated Database With Security Gaps

Posted: 3:52 am ET
[twitter-follow screen_name=’Diplopundit’ ]

 

According to the Privacy Impact Assessment (PIA) of December 2009, the Consular Consolidated Database (CCD)  contained over 100 million visa cases and 75 million photographs, utilizing billions of rows of data, and has a current growth rate of approximately 35 thousand visa cases every day.  The 2010 Consular Consolidated Database (CCD) Privacy Impact Assessment (PIA) describes (pdf) the CCD as “one of the largest Oracle based data warehouses in the world that holds current and archived data from the Consular Affairs (CA) domestic and post databases around the world.”  The 2011 OIG report says that in 2010, the CCD contained over 137 million American and foreign case records and over 130 million photographs and is growing at approximately 40,000 visa and passport cases every day.

Related posts:

 

#

 

Advertisements

CCD: Report Says Visa Processing Systems Pose Significant Challenges; Also Face User-Friendlessness?

Posted: 12:02 am EDT
[twitter-follow screen_name=’Diplopundit’ ]

Via GAO

According to Commerce, international travelers contributed $220.6 billion to the economy and supported 1.1 million jobs in 2014. Processing visas for such travelers as efficiently and effectively as possible without compromising our national security is critical to maintaining a competitive and secure travel and tourism industry in the United States. Although State has historically struggled with the task of maintaining reasonable wait times for NIV interviews, it has undertaken a number of efforts in recent years that have yielded substantial progress in reducing such waits.

Significant projected increases in NIV demand coupled with consular hiring constraints and other challenges could hinder State’s ability to sustain this progress in the future—especially in countries where the demand for visas is expected to rise the highest. These challenges heighten the importance of systematically evaluating the cost and impact of the multiple measures State has taken to reduce interview wait times in recent years and leveraging that knowledge in future decision making. Without this, State’s ability to direct resources to those activities that offer the greatest likelihood of success is limited. Moreover, State’s future capacity to cope with rising NIV demand will be challenged by inefficiencies in its visa processing technology; consular officers and management officials at the posts we visited pointed to cumbersome user procedures and frequent system failures as enormous obstacles to efficient NIV processing. State’s Bureau of Consular Affairs recognizes these problems and plans a number of system enhancements; however, the bureau does not systematically collect input from consular officers to help guide and prioritize these planned upgrades. Without a systematic effort to gain the input of those who employ these systems on a daily basis, State cannot be assured that it is investing its resources in a way that will optimize the performance of these systems for current and future users.

giphy_daleks

Consular officers and managers at posts we visited identified current information technology (IT) systems as one of the most significant challenges to the efficient processing of NIVs. Consular officers in all 11 focus groups we conducted across the four posts we visited stated that problems with the Consular Consolidated Database (CCD) and the NIV system create significant obstacles for consular officers in the processing of NIVs.26 Specifically, consular officers and managers at posts stated that frequent NIV system outages and failures (where the system stops working) at individual posts, worldwide system outages of CCD, and IT systems that are not user friendly, negatively affected their ability to process NIVs.

— NIV system outages and failures at posts: Consular officers we spoke with in Beijing, Mexico City, New Delhi, and São Paulo explained that the NIV system regularly stops working. This results in a reduced number of adjudications (whether being performed at the interview window or, for an IWP applicant, at an officer’s desk) in a day. Notably, consular officers in 4 of the 11 focus groups reported having to stop work or re-adjudicate NIV applications as a result of these NIV system failures. In fact, during our visit to the U.S. Embassy in New Delhi in March 2015, a local NIV outage occurred, affecting consular officers’ ability to conduct adjudications. In January 2015, officers in Bogotá, Guadalajara, Monterrey, and Moscow—among the top 15 posts with the highest NIV applicant volume in 2014— experienced severe NIV performance issues—specifically an inability to perform background check queries against databases.

— Worldwide outages and operational issues of CCD: Since July 2014, two worldwide outages of CCD have impaired the ability of posts to process NIV applications. On June 9, 2015, an outage affected the ability of posts to run checks of biometric data, thus halting most visa printing along with other services offered at posts.27 According to State officials, the outage affected every post worldwide for 10 days. The system was gradually repaired, but it was not fully restored at all posts until June 29, 2015, exacerbating already increased NIV interview wait times at some posts during the summer high demand season.According to State notices, another significant outage of CCD occurred on July 20, 2014, slowing NIV processing worldwide until September 5, 2014, when CCD returned to full operational capacity.28 State estimated that from the start of operational issues on July 20 through late July, State issued approximately 220,000 NIVs globally— about half of the NIVs State anticipated issuing during that period. According to officials in State’s Bureau of Consular Affairs, Office of Consular Systems and Technology (CST), who are responsible for operating and maintaining CCD and the NIV system, consular officers were still able to collect NIV applicant information during that period; however, processing of applications was significantly delayed with an almost 2-week backlog of NIVs. In the U.S. Consulate in São Paulo, a consular management official reported that due to this outage, the post had a backlog of about 30,000 NIV applications, or approximately 9 days’ worth of NIV interviews during peak season. Consular officers in 8 out of the 11 focus groups we conducted identified a lengthy CCD outage as a challenge to the efficient processing of NIVs.

— IT systems are not user friendly: In 9 out of 11 focus groups, consular officers described the IT systems for NIV processing as not user friendly. Officers in our focus groups explained that some aspects of the system hinder their ability to quickly and efficiently process NIVs. These aspects include a lack of integration among the databases needed for NIV adjudications, the need for manual scanning of documentation provided by an applicant, and an absence of standard keyboard shortcuts29 across all IT applications that would allow users to quickly copy information when processing NIV applications for related applicants, to avoid having to enter data multiple times. Some consular officers in our focus groups stated that they could adjudicate more NIVs in a day if the IT systems were less cumbersome and more user friendly.

— Consular officers in Beijing and Mexico City and consular management at one post indicated that the NIV system appeared to be designed without consideration for the needs of a high volume post, which include efficiently processing a large number of applications per adjudicator each day. According to consular officers, the system is poor at handling today’s high levels of demand because it was originally designed in the mid- 1990s. Consular officers in São Paulo stated that under current IT systems and programs, the post may not be able to process larger volumes that State projects it will have in the future.

— State, recognizing the limits of its current consular IT systems, initiated the development of a new IT platform. State is developing a new system referred to as “ConsularOne,” to modernize 92 applications that include systems such as CCD and the NIV system. According to State, ConsularOne will be implemented in six phases, starting with passport renewal systems and, in phase five, capabilities associated with adjudicating and issuing visas (referred to as non-citizen services). However, CST officials have yet to formally commit to when the capabilities associated with non-citizen services are to be implemented. According to a preliminary CST schedule, the enhanced capabilities associated with processing NIVs are not scheduled for completion until October 2019. Given this timeline, according to State officials, enhancements to existing IT systems are necessary and are being planned.

State Does Not Systematically Obtain End User Input to Prioritize Improvement Efforts for Current IT Systems

Although consular officers and managers we spoke with identified CCD and the NIV system as one of the most significant challenges to the efficient processing of NIVs, State does not systematically measure end user (i.e., consular officers) satisfaction. We have previously reported that in order for IT organizations to be successful, they should measure the satisfaction of their users and take steps to improve it.30 The Software Engineering Institute’s IDEALSM model is a recognized approach for managing efforts to make system improvements.31 According to this model, user satisfaction should be collected and used to help guide improvement efforts through a written plan. With such an approach, IT improvement resources can be invested in a manner that provides optimal results.

Although State is in the process of upgrading and enhancing CCD and the NIV system, State officials told us that they do not systematically measure user satisfaction with their IT systems and do not have a written plan for improving satisfaction. According to CST officials, consular officers may voluntarily submit requests to CST for proposed IT system enhancements. Additionally, State officials noted that an IT stakeholder group comprising officials in State’s Bureau of Consular Affairs regularly meets to identify and prioritize IT resources and can convey end user concerns for the system.32 However, State has not collected comprehensive data regarding end user satisfaction and developed a plan to help guide its current improvement efforts. Furthermore, consular officers continued to express concerns with the functionality of the IT systems, and some officers noted that enhancements to date have not been sufficient to address the largest problems they encounter with the systems.

Given consular officers’ reliance on IT services provided by CST, as well as the feedback we received from focus groups, it is critical that State identify and implement feedback from end users in a disciplined and structured fashion for current and any future IT upgrades. Without a systematic approach to measure end user satisfaction, CST may not be able to adequately ensure that it is investing its resources on improvement efforts that will improve performance of its current and future IT systems for end users.

#

Snapshot: Nonimmigrant Visa (NIV) Forecast Through Fiscal Year 2019-18 Million

Posted: 12:56 am EDT
[twitter-follow screen_name=’Diplopundit’ ]

Via GAO:

Since 2012, the Department of State (State) has undertaken several efforts to increase nonimmigrant visa (NIV) processing capacity and decrease applicant interview wait times. Specifically, it has increased consular staffing levels and implemented policy and management changes, such as contracting out administrative support services. According to State officials, these efforts have allowed State to meet the goals of Executive Order (E.O.) 13597 of increasing its NIV processing capacity by 40 percent in Brazil and China within 1 year and ensuring that 80 percent of worldwide NIV applicants are able to schedule an interview within 3 weeks of State receiving their application. Specifically, State increased the number of consular officers in Brazil and China by 122 and 46 percent, respectively, within a year of the issuance of E.O. 13597. Additionally, according to State data, since July 2012, at least 80 percent of worldwide applicants seeking a tourist visa have been able to schedule an interview within 3 weeks.

Two key challenges—rising NIV demand and problems with NIV information technology (IT) systems—could affect State’s ability to sustain the lower NIV interview wait times. First, State projects the number of NIV applicants to rise worldwide from 12.4 million in fiscal year 2014 to 18.0 million in fiscal year 2019, an increase of 45 percent (see figure).

Screen Shot 2015-10-27

Given this projected NIV demand and budgetary limits on State’s ability to hire more consular officers at posts, State must find ways to achieve additional NIV processing efficiencies or risk being unable to meet the goals of E.O. 13597 in the future. Though State’s evaluation policy stresses that it is important for bureaus to evaluate management processes to improve their effectiveness and inform planning, State has not evaluated the relative effectiveness of its various efforts to improve NIV processing. Without conducting a systematic evaluation, State cannot determine which of its efforts have had the greatest impact on NIV processing efficiency. Second, consular officers in focus groups expressed concern about their ability to efficiently conduct adjudications given State’s current IT systems. While State is currently enhancing its IT systems, it does not systematically collect information on end user (i.e., consular officer) satisfaction to help plan and guide its improvements, as leading practices would recommend. Without this information, it is unclear if these enhancements will address consular officers’ concerns, such as having to enter the same data multiple times, and enable them to achieve increased NIV processing efficiency in the future.

 #

Was the Consular Consolidated Database (CCD) the main target of the twin hackers?

Posted: 1:27 am EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

In May 2015, a federal grand jury indicted twin brothers Muneeb and Sohaib Akhter, 23, of Springfield, Virginia, on charges of aggravated identity theft, conspiracy to commit wire fraud, conspiracy to access a protected computer without authorization, access of a protected computer without authorization, conspiracy to access a government computer without authorization, false statements, and obstruction of justice.  According to USDOJ, the brothers and coconspirators also devised a scheme to hack into computer systems at the U.S.  Department of State to access network traffic and to obtain passport information.  (See Twin Brothers and Co-Conspirators on Alleged Scheme to Hack State Dept to Obtain Passport Information).

The bothers pleaded guilty on June 26, 2015.   On October 2, the USDOJ announced that Muneeb Akhter was sentenced for accessing a protected computer without authorization, making a false statement and obstructing justice.  Muneeb Akhter was sentenced to 39 months in prison and Sohaib Akhter was sentenced to 24 months in prison.  Each man was also sentenced to three years of supervised release. Case title: USA v. Akhter et al.  Below is an excerpt from the announcement:

[T]he Akhter brothers and co-conspirators engaged in a series of computer intrusions and attempted computer intrusions against the U.S. Department of State to obtain sensitive passport and visa information and other related and valuable information about State Department computer systems.  In or around February 2015, Sohaib Akhter used his contract position at the State Department to access sensitive computer systems containing personally identifiable information belonging to dozens of co-workers, acquaintances, a former employer and a federal law enforcement agent investigating his crimes.

Sohaib Akhter later devised a scheme to ensure that he could maintain perpetual access to desired State Department systems.  Sohaib Akhter, with the help of Muneeb Akhter and co-conspirators, attempted to secretly install an electronic collection device inside a State Department building.  Once installed, the device could have enabled Sohaib Akhter and co-conspirators to remotely access and collect data from State Department computer systems.  Sohaib Akhter was forced to abandon the plan during its execution when he broke the device while attempting to install it behind a wall at a State Department facility in Washington, D.C.

Furthermore, beginning in or about November 2013, Muneeb Akhter was performing contract work for a private data aggregation company located in Rockville, Maryland.  He hacked into the company’s database of federal contract information so that he and his brother could use the information to tailor successful bids to win contracts and clients for their own technology company.  Muneeb Akhter also inserted codes onto the victim company’s servers that caused them to vote for Akhter in an online contest and send more than 10,000 mass emails to students at George Mason University, also for the purpose of garnering contest votes.

In or about October 2014, Muneeb Akhter lied about his hacking activities and employment history on a government background investigation form while successfully obtaining a position with a defense contractor.  Furthermore, in or about March 2015, after his arrest and release pending trial, Muneeb Akhter obstructed justice by endeavoring to isolate a key co-conspirator from law enforcement officers investigating the conspirators’ crimes.  Among other acts, Muneeb Akhter drove the co-conspirator to the airport and purchased a boarding pass, which the co-conspirator used to travel out of the country to the Republic of Malta.  When the co-conspirator returned to the United States, Muneeb Akhter continued to encourage the co-conspirator to avoid law enforcement agents.

One of the brothers was profiled by WaPo in 2014. Both brothers started college at 16 and they were George Mason’s youngest graduates in 2011. In 2012, the brothers received a $200,000 grant from the Defense Advanced Research Project Agency, or DARPA.

The details of this case are even more disturbing.  Under Count Eight  (Conspiracy to Access a Government Computer without Authorization).

60. The Bureau of Consular Affairs (hereinafter “Bureau”) is a division of the State Department, which administers laws, formulates regulations, and implements policies relating to consular services and immigration. It has physical offices in Washington, DC.

61. Passport Lockbox (hereinafter “Lockbox”) is a Bureau program that performs payment processing, scarming of applications, and initial data entry for US. passport applications. Lockbox has a computer database containing imaged passport applications associated with real individuals. The imaged passport applications in Lockbox’s database contain, among other things, a photograph of the passport applicant, as well as certain personal information including the applicant’s full name, date and place of birth, current address, telephone numbers, parent information, spouse’s name, and emergency contact information.

62. ActioNet, Inc. (hereinafter “ActioNet”) is a contractor that provided information technology support to the State Department. It has physical offices in Falls Church, Virginia, located in the Eastern District of Virginia.

63. From in or about October 2014 to in or about February 2015, SOHAIB AKHTER was a contract employee at ActioNet assigned to a position at the State Department as a Tier II Application Support Resource in the Data Engineering and Data Management Program within the Bureau.

64. Prior to accessing the Lockbox database, and throughout his tenure as a contractor with the State Department, SOHAIB AKHTER was made aware of and indicated he understood: (a) the confidential nature of the Lockbox database and the confidential personal data contained therein; (b) the information contained in the passport records maintained by the State Department pursuant to Lockbox is protected from unauthorized disclosure by the Privacy Act of 1974, 5 U.S.C. § 552a; and (c) passport applications maintained by the State Department in the Lockbox database should be accessed only in connection with an employee’s official government duties and not the employee’s interest or curiosity.

69. MUNEEB AKHTER and SOHAIB AKHTER, UCC-l, and other coconspirators known and unknown to the Grand Jury, engaged in a series of computer intrusions and attempted computer intrusions against the State Department to obtain sensitive passport and visa information and other related and valuable information about State Department computer systems.

70. SOHAIB AKHTER used his contract position at the State Department to search for and access sensitive passport information belonging to coworkers, acquaintances, a former employer, and federal agents investigating him for crimes alleged in this Indictment. After accessing sensitive passport information from State Department computers, SOHAIB AKHTER copied, saved, and shared this information with coconspirators.

71. SOHAIB AKHTER also attempted to use his access to State Department computer systems to create an unauthorized account that would enable him to access State Department computer systems undetected. SOHAIB AKHTER surreptitiously installed malicious programs onto State Department computer systems in order to execute his plan to create the backdoor login account.

72. SOHAIB AKHTER orchestrated a scheme to secretly install a physical device at a State Department building known as SA-17. Once installed, the device would enable SOHAIB AKHTER and coconspirators to collect data from and remotely access State Department computer systems.

73. SOHAIB AKHTER led the conspiracy, organized the intrusion to install the physical device, recruited coconspirators to assist in execution of the intrusion, and managed the execution of the intrusion.

74. MUNEEB AKHTER provided technical assistance to SOHAIB AKHTER for the unauthorized access. MUNEEB AKHTER programmed the physical device, known as a “gumstix,” so that it would collect data from State Department computers and transmit it wirelessly to computers controlled by MUNEEB AKHTER and SOHAIB AKHTER and coconspirators.

75. On the day the scheme was executed, UCC-1 transported materials, including the gumstix, from MUNEEB AKHTER, located at the AKHTER residence, to SOHAIB AKHTER, located at SA-17.
[…]
78. In or about October 2014, SOHAIB AKHTER was hired by ActioNet to perform contract work for the State Department at both ActioNet offices in Falls Church, Virginia, and Bureau offices in Washington, DC.

79. Beginning on or about February 12, 2015, and continuing thereafter until on or about February 19, 2015, in Falls Church, Virginia, in the Eastern District of Virginia, and elsewhere, SOHAIB AKHTER, while employed at ActioNet, accessed the Lockbox database without authorization. .

80. Between on or about February 12, 2015, and on or about February 19, 2015, SOHAIB AKHTER conducted approximately 119 searches for U.S. passport records using the Passport Lockbox Lookup report. He accessed personal passport information for approximately 62 different individuals, including: G.R., a DHS special agent investigating the crimes alleged in this Indictment; UCC-1; A.I.; A.M., the CEO of Victim Company 2; and himself. In addition, SOHAIB AKHTER attempted to access passport information for S.T., a DHS special agent investigating the crimes alleged in this Indictment.

82. In or about February 2015, SOHAIB AKHTER viewed and copied from State Department computer systems the personal passport information associated with several individuals, including DHS Special Agent G.R.

83. In or about March 2015, MUNEEB AKHTER told UCC-1 that he and SOHAIB AKHTER stored the personal passport information that SOHIAB AKHTER removed from State Department systems on an external hard drive. MUNEEB AKHTER told UCC-1 that Special Agent G.R.’s information would be valuable to criminals on the “dark net” and that he was considering selling the information.

84. In or about February 2015, SOHAIB AKHTER downloaded several programs to a State Department computer. These programs included malicious software, or malware, which SOHAIB AKHTER hoped would enable him to access State Department computers remotely.

85. In or about February 2015, SOHAIB AKHTER told UCC-1 that if he was able to gain remote access to State Department computer systems, he could: access information on individuals’ passport applications; access and unilaterally approve visa applications without State Department authorization in exchange for payment; and create passports and visas and sell them on the “dark net.”

86. On or about February 15, 2015, SOHAIB AKHTER called UCC-1 and asked him to buy a drill. UCC-1 purchased the drill and then, pursuant to SOHAIB AKHTER’s request, drove to the AKHTER residence to pick up additional items from MUNEEB AKHTER. At the AKHTER residence, in Springfield, Virginia, in the Eastern District of Virginia, MUNEEB AKHTER told UCC-1 that he was programming a SD card, which was later to be inserted into the gumstix. MUNEEB AKHTER gave UCC-1 a bag containing a screwdriver, tape, glue, and the gumstix. Pursuant to SOHAIB AKHTER’s request, UCC—l drove to SA-17, in Washington, DC, and delivered the bag and items to SOHAIB AKHTER outside SA-17. Later that day, MUNEEB AKHTER drove separately to Washington, DC, and delivered the SD card to SOHAIB AKHTER.

87. On or about the evening of February 15, 2015, SOHAIB AKHTER called MUNEEB AKHTER and told him that he attempted to install the gumstix behind a wall inside SA-17 but was ultimately unsuccessful.

88. On or about February 19, 2015, SOHAIB AKHTER sent an email from his State Department email account to the email address akhters3@vcu.edu containing lines of code and headers for State Department servers.

#

We’re not sure reading this if the intrusion was done on the State Department’s Travel Document Issuance System (TDIS) which includes information from U.S. citizens and nationals applying for passports, other Department of State computer systems, passport acceptance agents, the Social Security Administration, the lockbox provider (CITIBANK), passport specialists, and fraud prevention managers, or, if the intrusion occurred on the Passport Information Electronic Records Systems (PIERS), or wait … the motherload, the Consular Consolidated Database (CCD) The Passport Lockbox program cited in the indictment is vague; it’s not a system of record according to the State Department’s System of Records Notices.  But the indictment identifies it as a State Department database. Could this be in reference to the Citibank® Lockbox Services? That is a high-speed processing environment and image-based platform for receivables management, advanced reporting and image inquiry used by the State Department to enable the scanning of applications, extraction of applicant photos received at lockbox locations and storing and batching of images.

Note that #69 of the indictment also alleges “a series of computer intrusions and attempted computer intrusions against the State Department to obtain sensitive passport and visa information;” does that mean the targeted system was the CCD?  The CCD provides access to passport data in Travel Document Issuance System (TDIS), Passport Lookout Tracking System (PLOTS), and Passport Information Electronic Records System (PIERS).  As of December 2009, the CCD also contains over 100 million visa cases and 75 million photographs, utilizing billions of rows of data, and has a current growth rate of approximately 35 thousand visa cases every day.

By the way, one of the brothers was a contract employee assigned to a position at the State Department as a Tier II Application Support Resource in the Data Engineering and Data Management Program within the CA Bureau from October 2014 to in or about February 2015 (#63).  In November 2014, the State Department suffered some “technical difficulties.” See State Dept Re-attached to the Internet, and About Those “Unrelated” Embassy Outages; State Department’s “Technical Difficulties” Continue Worldwide, So What About the CCD?

Was it just a coincidence that a master of the universe hacker was working at the State Department at the time when the agency’s systems were having technical difficulties?

Or were the Akhter twins the “technical difficulties”?

#

 

 

   

State Dept Authorization Bill Mandates Security Breach Reporting, NSA Consultations –Can PenTest Be Far Behind?

Posted: 12:27 am EDT
Updated: 11:23 am PDT
[twitter-follow screen_name=’Diplopundit’ ]

 

Update: A source on the Hill alerted us that the State Authorization bill was offered as an amendment when the NDAA was debated in the Senate last month but it was not voted on and the NDAA passed on June 18 (That would be H.R. 1735 which passed 215 (71-25)  We understand that both chambers are now starting the process to bring the bill to conference in order to resolve differences.  The State Authorization bill, we are told, will not be part of those discussions.  In order for this to move forward, it will either need to be brought to the floor as a stand alone vote or Corker/Cardin could try again to attach it to another piece of legislation. Given that this is the first authorization bill passed by the SFRC in 5 years, and made it through the committee with bi-partisan support, we suspect that the senators will not just easily forget about this. — DS

On June 9, 2015, U.S. Senators Bob Corker (R-Tenn.) and Ben Cardin (D-Md.), the chairman and ranking member of the Senate Foreign Relations Committee, applauded the unanimous committee passage of the Fiscal Year 2016 Department of State Operations Authorization and Embassy Security Act. The SFRC statement says that it has been five years since the Senate Foreign Relations Committee passed a State Department Authorization bill and 13 years since one was enacted into law.  This State Department Authorization bill has been offered as an amendment to the National Defense Authorization Act, which currently is on the Senate floor. It is quite lengthy so we’re doing this in installments.

Below is the section on information technology system security that mandates security breach reporting, as well as making State Dept systems and networks available to the Director of the National Security Agency (NSA) and any other such departments or agencies to carry out necessary tests and procedures.

The State Department’s Consular Consolidated Database (CCD) as of 2011 contains over 137 million American and foreign case records and over 130 million photographs and is growing at approximately 40,000 visa and passport cases every day. If the CCD is compromised, it would be a jackpot for hackers that would make the OPM hack severely pales in comparison.

If this bill passes, will the penetration test by NSA on one of the world’s largest data warehouses finally happen?

Via govtrack:

Section 206.Information technology system security

(a)In general

The Secretary shall regularly consult with the Director of the National Security Agency and any other departments or agencies the Secretary determines to be appropriate regarding the security of United States Government and nongovernment information technology systems and networks owned, operated, managed, or utilized by the Department, including any such systems or networks facilitating the use of sensitive or classified information.

(b)Consultation

In performing the consultations required under subsection (a), the Secretary shall make all such systems and networks available to the Director of the National Security Agency and any other such departments or agencies to carry out such tests and procedures as are necessary to ensure adequate policies and protections are in place to prevent penetrations or compromises of such systems and networks, including by malicious intrusions by any unauthorized individual or state actor or other entity.

(c)Security breach reporting

Not later than 180 days after the date of the enactment of this Act, and every 180 days thereafter, the Secretary, in consultation with the Director of the National Security Agency and any other departments or agencies the Secretary determines to be appropriate, shall submit a report to the appropriate congressional committees that describes in detail—

(1)all known or suspected penetrations or compromises of the systems or networks described in subsection (a) facilitating the use of classified information; and

(2)all known or suspected significant penetrations or compromises of any other such systems and networks that occurred since the submission of the prior report.

(d)Content

Each report submitted under subsection (c) shall include—

(1)a description of the relevant information technology system or network penetrated or compromised;

(2)an assessment of the date and time such penetration or compromise occurred;

(3)an assessment of the duration for which such system or network was penetrated or compromised, including whether such penetration or compromise is ongoing;

(4)an assessment of the amount and sensitivity of information accessed and available to have been accessed by such penetration or compromise, including any such information contained on systems and networks owned, operated, managed, or utilized by any other department or agency of the United States Government;

(5)an assessment of whether such system or network was penetrated by a malicious intrusion, including an assessment of—

(A)the known or suspected perpetrators, including state actors; and

(B)the methods used to conduct such penetration or compromise; and

(6)a description of the actions the Department has taken, or plans to take, to prevent future, similar penetrations or compromises of such systems and networks.

#

Related Post:
S.1635: DOS Operations Authorization and Embassy Security Act, Fiscal Year 2016 – Security Clearance

State Department’s Visa Systems Now Operational at 165 of 220 Posts Worldwide

Posted: 1:56 am  EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

The State Department’s Consular Consolidated Database problems that affected travelers globally is is now back online at 165 of 220 visa issuance posts worldwide.  The latest update does not explain in details the cause of the glitch except to cite the hardware issue.  It also says that service was restored “using a redundant, secondary backup system and other sources.”  It does not explain what “other sources” mean but if it took at least 9 days to get that redundant, secondary back-up system to kick in, that’s not a very good system.

The Consular Affairs-issued FAQ asks how many people were affected by this outage? The answer it provides to this question is neither here nor there.  Folks, if you can’t answer your own question, please don’t include it.

According to travel.state.gov, the average visa applications processed every day worldwide is 50,000 x 9 days (June 9-19)=450,000 + 25,000 (half the average daily applications) x 4 days (June 22-25) = 100,000. Total number potentially affected 550,000.  Is that close enough?

The June 25 update says that if systems had been operating normally, posts would have issued approximately 540,000 visas since the outage started. Whoa! Help us out here. What kind of refusal/approval rates are we looking at here? That 540,000 figure is a little hinky because not all applicants who apply are issued visas. If it would have issued 540,000  visas, what would have been the total number of applicants?  Note that all of them must pay the visa fees. We estimate that the USG loss from this latest glitch is between $72 to $84 million (average daily applications globally x no. of days x $160 visa fee). Is that too low?

Meanwhile, StarrFMonline.com reported that the US Embassy in Accra, has “dismissed reports that it is ripping Ghanaians off by accepting visa fees in spite of the visa issuance imbroglio that has hit US embassies across the world.” The consular section chief  had to explain that “if anybody was refused a visa, that was because of the case and has nothing to do with our technical issues.”

*

On June 24, the Bureau of Consular Affairs reports that 50 posts, representing nearly 73 percent of its  nonimmigrant visa demand worldwide, are back online and issuing visas.  It also says that “posts overseas have issued more than 150,000 non-immigrant visas since June 9.” And that for context, if systems had been operating normally, posts would have issued approximately 450,000 visas during the June 9-23 timeframe.

On June 25, the Bureau of Consular Affairs reports that 165 posts, representing more than 85 percent of nonimmigrant visa demand worldwide, are now online and issuing visas.  The update says that if systems had been operating normally, posts would have issued approximately 540,000 visas since the outage started.

Via travel.state.gov, June 25 update:

Visa Systems Issues

  • The Bureau of Consular Affairs reports that 165 posts, representing more than 85 percent of our nonimmigrant visa demand worldwide, are now online and issuing visas. 

  • Posts overseas issued more than 82,000 visas on June 24. 

  • Posts overseas have issued more than 238,000 non-immigrant visas this week. For context, if systems had been operating normally, posts would have issued approximately 540,000 visas since the outage started. 

  • We will continue to bring additional posts online until connectivity with all posts is restored. All posts worldwide are now scheduling interviews with applicants, including with those who applied after the systems problems began on June 9.

  • We deeply regret the inconvenience to travelers who are waiting for visas, as well as their families and U.S. businesses that have been affected.

  • We continue to post updates to our website, travel.state.gov.

 

Q: Reports indicate that your backlog is 700,000 visas. Is this accurate?

No. While there is a large backlog of cases to clear, it never approached that level, and we have already made good progress issuing those visas. Many posts are working overtime this week and during the upcoming weekend, and we expect to eliminate the backlog in a week or less.


Q: How old is this equipment? And does the age of the equipment and the need to have so many repairs to the hardware mean that this equipment should have been replaced? Is this a funding issue at the base of it?

The hardware that impacted the biometrics system is several years old. The Department was working to move the biometrics system off of this hardware.

The operational requirements to keep this database running for domestic and overseas passport and visa issuances caused delays in upgrading the database according to our planned maintenance schedule.

We have been working to upgrade our systems over the past year.

We will move ahead with planned migration and systems upgrades as soon as we fully restore service.

Q: How did you restore service?

We restored service using a redundant, secondary backup system and other sources. That data allowed us to begin to re-connect posts to the affected portion of the system and synchronize biometric data. This system is running on newer hardware, and has a synchronized standby system in a different Department data center.

In parallel, we are continuing to restore data from backups and overseas post databases. This process is ongoing.

Q: Do you know whether this is equipment that was acquired directly by the State Department, or was this acquired through a third-party contractor?

The equipment was acquired by the Department of State.

Q: How many people were affected by this outage?

During the past two weeks, consular sections have continued to interview travelers who applied June 8 or earlier. Those posts reconnected to our system are now issuing visas for those applicants.

Q: How are cases being prioritized?

We continue to facilitate urgent cases for those individuals who need to travel imminently, and will continue to do so until the systems are normal.

We apologize to travelers and recognize that this has caused hardship to some individuals waiting for visas as well as families and employers.

Q: What about the foreign agricultural workers (H2A visa holders?)

More than 2,500 temporary or seasonal workers have been issued new visas in Mexico since last week.

We will continue to prioritize H-2 applicants as our systems return to normal, and issue as many approved cases as possible. However, we will not be able to process these as quickly as we typically do until our systems are functioning normally. We continue to ask that any employers with urgent needs contact the post which is processing their applicants and we will do everything we can to facilitate the cases.

We are no longer asking CBP to provide Port of Entry waivers, as we have now begun issuing visas at border posts.

Visa applicants, including agricultural workers, who have not received a visa should not report to the border. Please contact the nearest embassy or consulate.

Read more here.

CCD Visa Update: 22 Posts Accounting For About Half of the Global Visa Volume Now Reconnected

Posted: 1:23 am  EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

On June 19, the Capital Press covering agriculture news in the western states reported that after a nine-day delay, the State Department began issuing visas again for Mexican farmworkers stranded at the U.S.-Mexican border wanting to head north for jobs. Visas were reportedly issued June 17 for a majority of some 200 people headed to Washington state to work in cherry harvest and other tree fruit work, according to Dan Fazio, director of the Washington Farm Labor Association in Olympia. The WAFLA posted its request for waiver of the visa requirement online here (pdf).

The update on the CCD glitch late last week said that 750 seasonal workers were issued visas as well as 1500 individuals with humanitarian cases. We asked CA about this since even us found this a tad confusing.  The earlier announcement said that the hardware glitch made it impossible to issue visas at this time. We requested confirmation that the seasonal workers and humanitarian cases who were “issued visas” were actually issued visa waivers. And if that was not the case, how was it possible to issue visas to these applicants if the hardware issues had yet to be fixed?

The Bureau of Consular Affairs has posted a June 22 update on the Consular Consolidated Database problems.  The update below does not specifically answer our questions but it indicates some development.  Note that the 22 posts are not identified in the update (which appears frustrating for consular clients venting on Facebook).  Visa applications affected are those submitted  on or after June 9.  With average global applications at 50,000 a day, this latest CCD glitch potentially affected about 450,000 visa applicants worldwide from June 9 – June 19. Whoa! Is that like $72 million in the last two weeks alone?

Here are the top 10 NIV issuance posts from FY2014. So these ten posts presumably have already been reconnected. We’re missing the next 12 posts that are also back online; hopefully, all 172 posts will be back online before too long.

Top Nonimmigrant (Temporary) Visa Issuance Posts | FY2014

Top Nonimmigrant (Temporary) Visa Issuance Posts | FY2014 (click for larger view)

Update below via travel.state.gov:

Visa Systems Issues

  • The Bureau of Consular Affairs is in the process of resolving technical problems with our visa systems. Though some progress has been made, biometric data processing has not been fully restored.
  • Our team continues to work on this 24/7 to restore the systems to full functionality.
  • As of noon today, 22 posts have been reconnected, representing about half of the global nonimmigrant visa volume.
  • We will continue to bring additional posts online until connectivity with all posts is restored.
  • Last week, nearly 1,250 temporary or seasonal workers who had been issued visas in the past were issued new visas in Mexico, and we have issued more than 3,000 visas globally for urgent and humanitarian travel.
  • We deeply regret the inconvenience to travelers and recognize the hardship to those waiting for visas, and in some cases, their family members or employers in the United States.
  • We continue to post updates to our website, travel.state.gov.

Q: How long before you restore full system functionality?

  • Public and private sector experts are making progress in correcting the visa problem, and we are striving to have the system fully reconnected sometime this week.
  • As of noon today, 22 posts have been reconnected, representing about half of the global nonimmigrant visa volume. All our servers appear to be stable. There is a large backlog of cases to clear, but we have already made good progress. We will continue to bring additional posts online until connectivity with all posts is restored.
  • Overseas and domestic passports are being issued.

Q: How many travelers are affected by this outage?

  • Most posts were able to handle visa interviews and some visa printing as usual through the end of last week. This week, many posts have rescheduled visa appointments.
  • We handle an average of 50,000 applications daily worldwide. Many applicants do not have immediate travel plans, and will receive visas in time for planned trips. We are prioritizing urgent medical and other humanitarian cases.

Q: Once operational, how will cases be prioritized?

  • We are already prioritizing urgent humanitarian cases and temporary agricultural workers. Once the systems are fully operational, we will work as quickly as possible to clear the backlog of pending visa cases.
  • We apologize to travelers and recognize that this has caused hardship to some individuals waiting for visas.

Q: What about domestic passports?

  • Domestic passport operations are functioning, with some processing delays. These technical problems have affected the intake of some mailed applications and same-day service at our passport agencies; however, we continue to issue routine and expedited passports to U.S. citizens for all overseas travel needs.

Q: What about overseas passports?

  • Overseas passport applications are being processed. There have been delays in some cases, but posts overseas are able to issue emergency passports in urgent cases.

Q: What about the foreign agricultural workers (H2A visa holders?)

  • We issued nearly 1,250 H-2 visas for agricultural and temporary workers last week. These are applicants whose biometric data was captured before the systems went down.
  • Additionally over the weekend we worked with DHS/CBP to facilitate the entry of more than 250 H-2 workers to travel to their places of employment in the United States.
  • CBP has agreed to give cases processed through the U.S. Department of State due consideration. However, no visa applicant, including agricultural workers, should report to the border without first having applied for a visa and having been processed through an embassy or consulate.
  • Based on this weekend’s systems progress, we are currently rescheduling more than 1,500 H-2 visa applicants who were unable to be interviewed last week because of systems problems.

Related posts:

State Dept’s Overseas Passport and Visa Systems Hit By Glitch Again, Suspends Issuance.
State Dept’s Consular Database Problems Dash Plans, Dreams … Also Cost Up to $1M/Day in California

State Dept’s Consular Database Problems Dash Plans, Dreams … Also Cost Up to $1M/Day in California

Posted: 4:46 pm  EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

On June 12, we posted about the technical problems with the State Department’s overseas passport and visa systems.  Passport applications accepted overseas on or after May 26, 2015 are affected but emergency passports are available.  A hardware failure on June 9 halted the flow of biometric clearance requests from posts to the State Department’s Consular Consolidated Database (CCD). Individuals who submitted online applications or were interviewed for visas on or after June 9 are affected and are asked to reschedule appointments . No emergency visas available. See State Dept’s Overseas Passport and Visa Systems Hit By Glitch Again, Suspends Issuance.

On June 15, the Bureau of Consular Affairs posted the following update on its Facebook page but not on its travel.state.gov news page:

We continue to experience technical problems with our visa systems. This issue is not specific to any particular country or visa category. We apologize for the inconvenience and we are working around the clock to correct the problem. Currently, we are unable to print most immigrant and nonimmigrant visas approved after June 8, 2015. In addition, U.S. embassies and consulates are unable to process new applications submitted on or after June 9, 2015. If you have a visa interview appointment scheduled for June 14-20, 2015, and you submitted your DS-160 online application **after June 9, 2015,** you should reschedule your appointment. If you submitted your DS-160 online application prior to June 9, 2015, you should plan to attend your scheduled visa interview appointment. Our embassies and consulates will be posting location-specific information on their websites, so please check the website of the location where you applied for your visa for more information. 

The technical issues also affected the Department of State’s ability to adjudicate applications for U.S. passports accepted overseas between May 26 and June 14, 2015. If you applied for a U.S. passport overseas during this time frame and have travel plans within the next 10 business days, please consider requesting an emergency passport at the U.S. embassy or consulate at which you originally applied. Information about how to apply for an emergency passport is available on the embassy/consulate website.

The previous time the CCD crashed big time was last summer (see State Dept’s Critical National Security Database Crashes, Melts Global Travelers’ Patience).  It could just be a coincidence (or not!) but the crash has now happened twice during the peak travel season. During the meltdown last summer, CA said that CCD was going to have an upgrade at the end of 2014. It also  said at that time that the upgrade plan included two redundant systems. If this glitch started on May 26th, we’re approaching the three week-mark. And so far, those redundant systems are missing in action.

The Bureau of Consular Affairs on its FAQ states that “This is not the same issue as last year.” But we learned from an unofficial source that “All line officers know that last summer’s CCD glitch was never completely fixed.” 

So, which is it?

On June 15, the Wall Street Journal reported that the CCD glitch has left agricultural workers stranded at the border just as the summer harvest gets under way. Jason Resnick, the general counsel for the Western Growers Association, which represents farmers in California, Arizona and Colorado calls this glitch, “a crisis.” Apparently, more than 1,000 workers who expected H-2A agricultural visas are stuck on the Mexican side of the border, where motels are overflowing.

“The workers are overdue to start harvesting berries and other crops on U.S. farms. Mr. Resnick estimated that California agriculture, already stressed by drought, is losing $500,000 to $1 million for each day of delay.”

.

The State Department’s consular operation is an enormous one with many parts and affects a large number of travelers.  The State Department issued 9,932,480 nonimmigrant/temporary visas in FY2014. It issued 467,370 immigrant/permanent visas in FY2014. During the same period, domestic and overseas passport offices issued 14,087,341 U.S. passports (including 1,463,191 passport cards).

A small fraction of those frustrated travelers have taken to Facebook to connect with Consular Affairs.

One who is stuck in Canada and could not return to her 14th month old baby writes:

Hi, do you have any timeline to fix the issues? I live in Boston, US & visited Vancouver, BC consulate on June 10th for my visa interview. visa officer told me that your visa is approved and you will get your passport back in 3 days. However, since June 10th, there is no update on my visa. I also inquired with Vancouver Consulate and they are ready to give me passport back without visa. As a matter of fact, I can’t enter into US legally until and unless I got printed Visa. My 14th month baby is waiting in Boston,MA and I got stuck here. Can’t do anything.

Here’s one waiting to be reunited with a loved one after a long wait:

Waiting is excruciating my husband was approved on june 10th and my mother has passed away. We need him home please get this fixed our application has been in since 2013.

Somebody who is pregnant, stuck in Mexico writes:

I am currently 8 months pregnant and have been waiting for my TD visa renewal since mid May. Since I will soon have travel restrictions to fly back home, does this qualify as an urgent humanitarian situation where I should contact the embassy in Mexico directly?

One who missed grandma’s funeral makes a plea:”

Can you tell us a estimated time??? My case was expedited and I miss my Grandmother funeral. Me and my wife were supposed to travel yesterday. Please get this fixed.please

A family stuck in Mexico:

Do you have an ETA in order for ys to plan accordingly? I had my appointment on friday june 12th and I am stuck in Mexico (H1B renewal) without passport and without the ability to get back to work in Boston. Flight fees, hotel fees and a family of 4 that needs to get back to Boston.

Some people missing a convention:

My wife and I have a flight to ny tom. Our visa were approved on 9th. We are part of 100+ group attending a convention. Do we expect to get our passports with visas today?

Somebody stuck in Guatemala, fears loss of a job:

All my documents were in order and approved June 1st. my husband and I are in a dire situation stuck in Guatemala. I’m at risk of losing my job if I don’t return to USA.

A frustrated somebody who calls out other technical issues:

There’s always something wrong. There are already technical issues with payment of IV fees, DS260 and DS261. This system needs to be revamped. What’s the government doing about this?!

A Romanian group who worked and saved to attend the Genius Olympiad:

We are desperate. We have a plane ticker for tomorrow and we were supposed to go to an international competition (Genius Olympiad) in Oswego, NYC. Apart from the part that we lost thousands of dollars, our hopes got crashed because we worked for a year at our projects and invested a lot of time and monney… For… Nothing?!?!?! How come you have no plan B for solving this issue? We tried making an appointment more than one month ago and they said on the 9th on june will be our interview, we said it s too late for us but they said that the visa will be delivered within 2 days maximum. And here we are 5 days later with no visas, with crashed hopes, tons of lost monney, wasted time… Should i go on??

#

State Dept’s Overseas Passport and Visa Systems Hit By Glitch Again, Suspends Issuance

Posted: 11:09 pm EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

Somebody sent us a note on June 11 asking, “Do you think the Chinese hackers could fix the Consular Consolidated Database?” Fix, how, we asked the white, empty space of the burn bag email.

Today, this pops up:

.

Here is the information provided by the State Department to the public:

Passport/Visa Systems Errors

  • The Bureau of Consular Affairs is currently experiencing technical problems with our overseas passport and visa systems.
  • This issue is not specific to any particular country, citizenship document, or visa category.
  • The Consular Consolidated Database (CCD) problems we are experiencing are not the same challenges we overcame last summer. We are working urgently to correct the problems and restore our system to full operational status as soon as possible.
  • We apologize to applicants who are experiencing delays or are unable to obtain a passport overseas, Consular Report of Birth Abroad, or U.S. visa at this time. Domestic passport issuances are not affected at this time. We are able to issue emergency passports to U.S. citizens overseas for urgent travel.
  • We are seeking to assist non-immigrant visa applicants with urgent humanitarian travel. Travelers with an urgent humanitarian need for travel should contact their nearest U.S. embassy or consulate.
  • We are aware of pending overseas adoption cases, including in China. We are prioritizing these cases and seek to issue these visas with few delays.
  • We regret the inconvenience to travelers, and remain committed to facilitating legitimate travel while protecting our borders. We are working urgently to correct the problem and expect our system to be fully operational again soon.
  • We will post updates to Travel.State.Gov as more information becomes available.

How is this affecting consular operations?

      Passports

  • Passport applications accepted overseas on or after May 26, 2015 are affected. If you applied for a U.S. passport during this time frame and have travel plans within the next 10 business days, please consider requesting an emergency passport at the U.S. embassy or consulate at which you originally applied. Information about how to apply for an emergency passport is available on the website of the nearest U.S. embassy or consulate.

      Visas

  • A hardware failure on June 9 halted the flow of biometric clearance requests from posts to the Consular Consolidated Database (CCD). Individuals who submitted online applications or were interviewed for visas on or after June 9 may experience a delay in the processing.
  • The systems in place to perform required national security checks before we issue visas are experiencing technical difficulties. As a result, we are unable to print visas, regular passports overseas, and other travel documents.
  • We cannot bypass the legal requirements necessary to screen visa applicants before we issue visas for travel.
  • As a result, there is a backlog of visas waiting to be processed. We are working as quickly as possible to resolve the issue and to clear the backlog.
  • The technical issues we are experiencing have disrupted or prevented some of the Department’s primary data-share partners from accessing visa records.

The public notice notes that visas cannot be printed without using the CCD system as security measures prevent consular officers from printing a passport, report of birth abroad, or visa until the case completes the required national security checks.visas

On the CA Q&A whether this was a malicious action or hack, the public response only says that the State Department is “working urgently to correct the problem and expect the system to be fully operational again soon.”  There is currently no available timeline on when full system functionality may be restored.

Read the full notice here.

We should note that the person in charge of the CA Bureau’s response the last time the CCD had a meltdown was  Greg Ambrose, a career IT official who was the chief of consular systems and technology (State/CA/CST). FCW previously reported this:

He has been working on a modernization project at State that involves taking the Consular Consolidated Database, a massive system of 12 databases used to process passport and visa applications, from Windows 2003 to Linux. He is also moving the data warehouse to the more powerful Oracle 11g platform. The goal is to give the stovepiped legacy systems a single look and feel.

Not this time around.

.

.

Citing internal State Department email, FCW says that Mr. Ambrose is scheduled to leave his CST job on June 11.  As of today, Kenneth Reynolds, Ambrose’s deputy reportedly replaced him on an acting capacity.

#

Related posts:

 

 

State Department’s “Technical Difficulties” Continue Worldwide, So What About the CCD?

— Domani Spero
[twitter-follow screen_name=’Diplopundit’ ]

 

The “technical difficulties” at the State Department continue today.  State Department spokesman Jeff Rathke told Yahoo News that  the State Department is still investigating who — or what — launched the attack saying, “I don’t have anything to share at this point on the origins of the intrusion.”

Rathke said the attack only hit unclassified email systems at the State Department — and not business databases that contain information about Americans or, for example, foreign visa applicants. Although the temporary shutdown was previously scheduled, “in this case, the response to this specific incident needed to be more comprehensive than our regular updates.

Congress is apparently interested on what’s going on.

Meanwhile, the Department’s mobile site go.state.gov remains down, and the “technical difficulties” now include, according to tweets from overseas posts, not just inability to use email  but also inability to accept credit card payment for visa and passport services, and unusable contact forms for visa and passport inquiries.


US Embassy Albania


US Embassy London

 

 

U.S. Embassy Manila

U.S. Embassy Beirut

 

US Embassy Turkey

U.S. Embassy Moscow

 

U.S. Embassy Madrid

* * *

Below is the template of the notice used today:

U.S. embassies and consulates are currently experiencing technical difficulties that may result in delays in visa processing and receiving and sending communications. Additionally, applicants who have interviews for student and exchange visitor (F/M/J) visas scheduled for this week should bring proof of payment of the SEVIS fee. U.S. citizens may also experience delays in sending and receiving communications. U.S. citizens requiring emergency assistance should contact the Embassy [INSERT contact info].

 

We doubt if the State Department would have acknowledged this intrusion had the Associated Press not reported it on Sunday. On a related matter, we understand that Consular Affairs’ Consular Consolidated Database has been having problems “lately.”

Can somebody please ask CA if these ongoing problems are related to the technical difficulties from this past summer, or if this is related to the just known intrusion that brought down the email system and the GO site? We’re not terribly technical but curious — if a cyber intruder starts deleting data from the CCD, would anyone notice what’s missing?

* * *