Tag Archives: glitch

State Dept’s Consular Database Problems Dash Plans, Dreams … Also Cost Up to $1M/Day in California

June 16, 2015 By in Consular Work, Foreign Service, Functional Bureaus, Security, Shutdown, Social Media, State Department, Technology and Work, Trends, Visas Tags: , , , , , , , ,

Posted: 4:46 pm  EDT
[twitter-follow screen_name=’Diplopundit’ ]

 

On June 12, we posted about the technical problems with the State Department’s overseas passport and visa systems.  Passport applications accepted overseas on or after May 26, 2015 are affected but emergency passports are available.  A hardware failure on June 9 halted the flow of biometric clearance requests from posts to the State Department’s Consular Consolidated Database (CCD). Individuals who submitted online applications or were interviewed for visas on or after June 9 are affected and are asked to reschedule appointments . No emergency visas available. See State Dept’s Overseas Passport and Visa Systems Hit By Glitch Again, Suspends Issuance.

On June 15, the Bureau of Consular Affairs posted the following update on its Facebook page but not on its travel.state.gov news page:

We continue to experience technical problems with our visa systems. This issue is not specific to any particular country or visa category. We apologize for the inconvenience and we are working around the clock to correct the problem. Currently, we are unable to print most immigrant and nonimmigrant visas approved after June 8, 2015. In addition, U.S. embassies and consulates are unable to process new applications submitted on or after June 9, 2015. If you have a visa interview appointment scheduled for June 14-20, 2015, and you submitted your DS-160 online application **after June 9, 2015,** you should reschedule your appointment. If you submitted your DS-160 online application prior to June 9, 2015, you should plan to attend your scheduled visa interview appointment. Our embassies and consulates will be posting location-specific information on their websites, so please check the website of the location where you applied for your visa for more information. 

The technical issues also affected the Department of State’s ability to adjudicate applications for U.S. passports accepted overseas between May 26 and June 14, 2015. If you applied for a U.S. passport overseas during this time frame and have travel plans within the next 10 business days, please consider requesting an emergency passport at the U.S. embassy or consulate at which you originally applied. Information about how to apply for an emergency passport is available on the embassy/consulate website.

The previous time the CCD crashed big time was last summer (see State Dept’s Critical National Security Database Crashes, Melts Global Travelers’ Patience).  It could just be a coincidence (or not!) but the crash has now happened twice during the peak travel season. During the meltdown last summer, CA said that CCD was going to have an upgrade at the end of 2014. It also  said at that time that the upgrade plan included two redundant systems. If this glitch started on May 26th, we’re approaching the three week-mark. And so far, those redundant systems are missing in action.

The Bureau of Consular Affairs on its FAQ states that “This is not the same issue as last year.” But we learned from an unofficial source that “All line officers know that last summer’s CCD glitch was never completely fixed.” 

So, which is it?

On June 15, the Wall Street Journal reported that the CCD glitch has left agricultural workers stranded at the border just as the summer harvest gets under way. Jason Resnick, the general counsel for the Western Growers Association, which represents farmers in California, Arizona and Colorado calls this glitch, “a crisis.” Apparently, more than 1,000 workers who expected H-2A agricultural visas are stuck on the Mexican side of the border, where motels are overflowing.

“The workers are overdue to start harvesting berries and other crops on U.S. farms. Mr. Resnick estimated that California agriculture, already stressed by drought, is losing $500,000 to $1 million for each day of delay.”

.

The State Department’s consular operation is an enormous one with many parts and affects a large number of travelers.  The State Department issued 9,932,480 nonimmigrant/temporary visas in FY2014. It issued 467,370 immigrant/permanent visas in FY2014. During the same period, domestic and overseas passport offices issued 14,087,341 U.S. passports (including 1,463,191 passport cards).

A small fraction of those frustrated travelers have taken to Facebook to connect with Consular Affairs.

One who is stuck in Canada and could not return to her 14th month old baby writes:

Hi, do you have any timeline to fix the issues? I live in Boston, US & visited Vancouver, BC consulate on June 10th for my visa interview. visa officer told me that your visa is approved and you will get your passport back in 3 days. However, since June 10th, there is no update on my visa. I also inquired with Vancouver Consulate and they are ready to give me passport back without visa. As a matter of fact, I can’t enter into US legally until and unless I got printed Visa. My 14th month baby is waiting in Boston,MA and I got stuck here. Can’t do anything.

Here’s one waiting to be reunited with a loved one after a long wait:

Waiting is excruciating my husband was approved on june 10th and my mother has passed away. We need him home please get this fixed our application has been in since 2013.

Somebody who is pregnant, stuck in Mexico writes:

I am currently 8 months pregnant and have been waiting for my TD visa renewal since mid May. Since I will soon have travel restrictions to fly back home, does this qualify as an urgent humanitarian situation where I should contact the embassy in Mexico directly?

One who missed grandma’s funeral makes a plea:”

Can you tell us a estimated time??? My case was expedited and I miss my Grandmother funeral. Me and my wife were supposed to travel yesterday. Please get this fixed.please

A family stuck in Mexico:

Do you have an ETA in order for ys to plan accordingly? I had my appointment on friday june 12th and I am stuck in Mexico (H1B renewal) without passport and without the ability to get back to work in Boston. Flight fees, hotel fees and a family of 4 that needs to get back to Boston.

Some people missing a convention:

My wife and I have a flight to ny tom. Our visa were approved on 9th. We are part of 100+ group attending a convention. Do we expect to get our passports with visas today?

Somebody stuck in Guatemala, fears loss of a job:

All my documents were in order and approved June 1st. my husband and I are in a dire situation stuck in Guatemala. I’m at risk of losing my job if I don’t return to USA.

A frustrated somebody who calls out other technical issues:

There’s always something wrong. There are already technical issues with payment of IV fees, DS260 and DS261. This system needs to be revamped. What’s the government doing about this?!

A Romanian group who worked and saved to attend the Genius Olympiad:

We are desperate. We have a plane ticker for tomorrow and we were supposed to go to an international competition (Genius Olympiad) in Oswego, NYC. Apart from the part that we lost thousands of dollars, our hopes got crashed because we worked for a year at our projects and invested a lot of time and monney… For… Nothing?!?!?! How come you have no plan B for solving this issue? We tried making an appointment more than one month ago and they said on the 9th on june will be our interview, we said it s too late for us but they said that the visa will be delivered within 2 days maximum. And here we are 5 days later with no visas, with crashed hopes, tons of lost monney, wasted time… Should i go on??

#

Related posts:

 

State Dept/CBP Reportedly Announced Fix for Certain Applicants Ensnared By Visa Glitch

August 12, 2014 By in Consular Work, Federal Agencies, Functional Bureaus, State Department, Technology and Work, Visas Tags: , , , , , , ,

— Domani Spero
[twitter-follow screen_name=’Diplopundit’ ]

 

The State Department’s Consular Consolidated Database has been having performance issues since July 19th. We have written about it in this blog (see State Dept Answers FAQ on Ongoing Visa and Passport Database Performance Issues and  State Dept’s Critical National Security Database Crashes, Melts Global Travelers’ Patience).

Last week, Greenberg Traurig posted on The National Law Review that the State Department and the U.S. Customs and Border Protection (CBP) have reportedly announced a fix for certain visa applicants affected by the technical glitch.

“DOS and CBP will, on a case-by-case basis, waive nonimmigrant visa (H-1B, L-1, O-1, etc.) requirements for admission into the United States. In particular, applicants whose U.S. travel involves an “emergency” (i.e., humanitarian travel and life-and-death situations) or impacts U.S. national interests may request consideration for special travel permission.”

The post further states that if “emergency” travel is approved, the embassy or consulate will issue a transportation letter for presentation to common carriers to allow boarding of international U.S.-bound flights. (See DOS and CBP Announce Fix for Certain Visa Applicants Who Are Experiencing Consular Delays Due to Recent Technical Challenges).

This information is nowhere to be found on the State Department’s website or on the Visa Section of travel.state.gov nor the FB page of the Bureau of Consular Affairs. No such announcement is made available from the CBP website.

An  August 10 update from U.S.-China Visa Law Blog includes the following details:

A nonimmigrant visa applicant whose U.S. travel is urgent because it either involves an “emergency” or impacts U.S. national interests, may request consideration for special travel permission to the United States if their visa issuance is delayed as a result CCD systems problems. “Emergencies” in this instance include urgent humanitarian travel and life-and-death situations. Upcoming business engagements and U.S. employment needs are “not typically considered humanitarian emergencies and likely will not be considered as such in most cases.”

If approved jointly by the State Department and U.S. Customs and Border Protection (CBP), the consular post that accepted the visa application will release the traveler’s passport and will issue a transportation letter, which can be presented to the airlines to allow boarding of international U.S.-bound flights. Upon arrival to a U.S. port of entry and presentation of the transportation letter, CBP will waive the nonimmigrant visa requirement for admission.

Read more:  An Computer Crash Hobbles U.S. Visa, Passport Operations in China (Aug. 10 Update).

It is, of course, just a coincidence that the two sources noting the transportation letter fix are both law firms working on immigration, right? 😉  CA bureau’s FB page does not have an August 8 or August 10 update that includes this information. If there was an announcement, are we to understand that it was done on limited distribution with the State/CBP telling lawyers about this but not releasing this guidance to the general public?

We must confess that we’ve made a mistake of asking for clarification about this from the press office of the Bureau of Consular Affairs.  It turns out that some  folks there are unable to answer “yes” or “no” questions and are only able to provide cut and paste “on background” information for recycled details already publicly available.

Don’t get us wrong. It certainly is impressive cut and paste skills, but we won’t help them recycle the canned info and add to the glut.

* * *

 

 

Related articles

 

 

 

 

 

 

 

 

 

State Dept Answers FAQ on Ongoing Visa and Passport Database Performance Issues

July 31, 2014 By in Consular Work, Contractors, Foreign Service, Functional Bureaus, Social Media, Spectacular, State Department, Technology and Work, U.S. Missions, Visas Tags: , , , , , , , 2 Comments

— Domani Spero
[twitter-follow screen_name=’Diplopundit’ ]

 

Yesterday, we posted about the troubled Consular Consolidated Database (CCD) (see State Dept’s Critical National Security Database Crashes, Melts Global Travelers’ Patience).  During the Daily Press Briefing, yesterday, the State Department officially stated that it believed the root cause of the problem was “a combination of software optimization and hardware compatibility issues.” According to the deputy spokesperson, the servers are getting back online but that they are coming back in a queue and that fixes are not being done on a country-by-country basis. And by the way, it’s not just the peak summer travel season, there’s also the Africa Summit in D.C. next week.

“Obviously, there’s actually a huge crush right now because of the Africa Leaders Summit, so obviously that’s a huge priority for us to make sure everybody gets their visas for the Africa Leaders Summit. We do believe that a vast majority of the travelers who have applied for visas for the summit have been issued.”

CA’s FB folks have been regularly answering questions from angry complaints posted on its Facebook page and have announced that they will continue to monitor and respond to consular clients at 9:00 EDT tomorrow, Thursday, July 31.

Late yesterday, the Bureau of Consular Affairs also posted a new Frequently Asked Questions on Facebook and on its website (not easily accessible from the main visa page) concerning the CCD performance issues and the steps taken to address those issues. Perhaps the most surprising is that its back-up capability and redundancy built into the CCD were both affected killed by the upgrade that hobbled the system.  Something to look forward to by end of calendar year — CA is upgrading the CCD to a newer version of the Oracle commercial database software and that plan includes establishing two fully redundant systems. We are republishing the FAQ in full below.

Screen Shot 2014-07-31 at 12.07.05 AM

Information Regarding Ongoing Consular Consolidated Database (CCD) Performance Issues and Steps Taken| JULY 30, 2014

The Department of State continues to work to restore our visa system to full functionality.

We anticipate it will take weeks to resume full visa processing capacity.

We continue to prioritize immigrant visas, including adoption cases.  So far, we have been able to issue most cases with few delays.

Nearly all passports are currently being issued within our customer service standards, despite the system problems.

We are able to issue passports for emergency travel.

Frequently Asked Questions

Q:  What caused the system performance issues?  Hardware, software, or both?  Details?

On July 20, to improve overall system performance and address previous intermittent performance issues, we updated software as recommended. Our database began experiencing significant performance issues shortly after this maintenance was performed.

A root cause has not been identified at this time.  Current efforts are focused on bringing the system back to normal operations.  Once that has been accomplished, resources will be applied to determine the root cause.

Q:  What steps did we take to mitigate the performance issues?

Since July 20, our team has worked to restore operations to full capacity.  On July 23, the Consular Consolidated Database (CCD) was brought back online with limited capacity.

The Department of State is working with Oracle and Microsoft to implement system changes aimed at optimizing performance and addressing ongoing performance issues.

We are incrementally increasing the number of processed cases as our systems will allow.

Q:  Has the Bureau of Consular Affairs experienced these types of outages in the past?

CA has experienced minor outages in the past, but never of this magnitude.  We have a plan in place to mitigate these occurrences in the future.

Q:  Is the software to blame?  Are contractors at fault?  Why was this allowed to happen?

We have been working to improve our services through upgrades while maintaining existing operations worldwide.  However, we are limited by outdated software and hardware.

Q:  Why did those steps not work?  What’s the next step?

We have not determined why the problems occurred.  We are working with our contractor and the software vendor to address the problems.

We are bringing additional servers online to increase capacity and response time.

Q: Why wasn’t there a back-up server?

There was back-up capability and redundancy built into the system.  However, the upgrade affected not only our current processing capability, but also our ability to use our redundant system.

Q: What steps are being taken prevent this from happening again?

CA has a plan in place to upgrade the CCD to a newer version of the Oracle commercial database software by the end of the calendar year.  We are working to ensure the existing system will remain fully functional until the new database is up and running and thoroughly tested.  The plan includes establishing two fully redundant systems.

Q:  If CA is fee funded, why can’t it build a robust database that doesn’t fail?

The database has grown dramatically, in both quantity of data and functionality, and vastly improved border security.  In addition to checking names against databases, we review fingerprints and perform facial recognition.

We are working towards modernization of our software, hardware, and infrastructure.  Demand for our services outpaced our modernization efforts.

Consular Affairs has, and has had, a redundant system.  However, the upgrade affected not only our current processing capability, but also our ability to use our redundant system.  This is one of the issues we are urgently addressing now.

Q:  What do I need to know if I’m a passport applicant?

Almost all passports are currently being issued within our customer service standards, despite the system problems.

We are able to issue passports for emergency travel.

Q:  What do I need to know if I’m a visa applicant?

Visa applicants they can expect delays as we process pending cases.  We remain able to quickly process emergency cases to completion.

We are working urgently to correct the problem to avoid further inconveniencing travelers.

We are posting updates to the visa page of travel.state.gov, and our embassies and consulates overseas are communicating with visa applicants.

In addition to communicating through our websites, e-mail, and letters, we are also reaching out to applicants via Facebook and other social media sites, such as Weibo, to relay the latest information.

Q:  Why hasn’t the Department been more forthcoming until now?

We have experienced CCD outages in the past, but they have never disrupted our ability to perform consular tasks at this magnitude.

We informed the public as soon as it was apparent there was not a quick fix to bring the CCD back to normal operating capacity, and are briefing Congressional staffers regularly.

Q:  What is the outlook for Non-immigrant visas?  When do we estimate the backlog will be processed?

That will depend on a number of factors.  Current efforts are focused on bringing the system back to normal operations.

We must also continue processing new requests.  We are committed to reducing the number of pending visa cases as quickly as possible, but we want applicants to know that we will continue to be operating at less than optimal efficiency until the system is restored to full functionality.

Q:  Is the Department going to reimburse applicants who missed flights/canceled weddings/missed funerals?

We sincerely regret any delays, inconvenience, or expense that applicants have may have incurred due to the CCD performance issues.

While it might be of little solace to those who have experienced hardship, we are always very careful to tell travelers NOT to make travel plans until they have a visa in hand.  Even when the CCD is operating normally, there may be delays in printing visas.

The Department does not have the authority to reimburse applicants for personal travel, nor do we include these costs when calculating our fees.  The Department cannot refund visa fees except in the specific circumstances set out in our regulations.

Q:  What impact will this have on SIVs?

We have the highest respect for the men and women who take enormous risks in supporting our military and civilian personnel.  We are committed to helping those who have helped us.  While issuances of Special Immigrant Visas (SIVs) to Afghans and Iraqis have been impacted, as have visa issuances around the world, SIV processing continues and remains a high priority.

Q: How is this impacting student visas?  They are scheduled to start the fall semester soon.

We are committed to issuing visas to all qualified students and exchange visitors.  Issuance of student and exchange visitor visas has been impacted in the past few days, but visa processing continues.

We understand the importance to international students and exchange visitors, their families, and their U.S. host institutions of timely visa issuance in order to facilitate travel and to ensure all students and exchange visitors may begin their programs on time.

Q: What about situations where the student won’t arrive to school on time?

Students should contact their educational institution’s Designated School Official (F and M visas) or designated U.S. sponsor’s Responsible Officer (J visas) and discuss with them what arrangements they can provide for you to begin your program after the start date on your Form I-20 (F and M visas) or Form DS 2019 (J visas), should such a circumstance become necessary.

Q:  Will this have any impact on the Diversity Visa program in September?

While issuances of all immigrant visas, including diversity visas, have been impacted in the past few days, IV processing continues and remains a high priority.  The Department expects to have used all numbers for DV-2014 when the program year ends on September 30, 2014.

Q:  What impact do we anticipate this will have on the U.S. economy?

Tourism and students have a major impact on our economy.  Last year, it was estimated that international visitors spent $180.7 billion and supported 1.3 million American jobs.  International students contribute $24.7 billion to the U.S. economy through their expenditures on tuition and living expenses, according to the Department of Commerce.

We recognize the significant impact that international travel and tourism has on the U.S. economy, and are taking all possible steps to ensure that the economic impact is minimal.

People traveling under the Visa Waiver Program are not affected at all; nor are those whose previously-issued visas remain valid.

We routinely advise applicants needing new visas to make appointments well in advance of their planned travel, and not to book their travel until they have their printed visas in hand.

The original post is available here.  If CA is reading this, it would be helpful if a link to the FAQ is posted on the main visa page of travel.state.gov and in the News section.  We were only able to find the FAQ from a link provided in Facebook and not from browsing around the travel.state.gov website.

 

 

 

Related articles