At noon on Thursday 7 May in Courtroom 12 at the Auckland High Court, Justice Geoffrey Venning was to rule on the injunction Vodafone had sought to delay the launch of Telecom’s 3G network.
He never delivered that decision.
Just hours before, the two carriers announced they had settled their differences out of court. The XT launch would be delayed – but only by 17 days. Enough time for Telecom to install filters on the cell sites that Vodafone claimed were transmitting rogue radio waves.
The previous day, in a courtroom packed with senior industry executives and technology journalists, Justice Venning had listened as Julian Miles QC put forward Vodafone’s case and Pheroze Jargose defended Telecom’s. At 4.30pm he returned to his chambers with the 17 sworn affidavits which had formed the basis of the lawyers’ arguments that day.
Telecommunications Review has a copy of all but two of those affidavits – those signed by Stuart Sutton and Gemma Roper are sealed – and is able to report on what led to the decision by Vodafone to take Telecom to court.
Were they right to do so? You be the judge…
From an engineering perspective New Zealand’s two mobile networks have coexisted harmoniously for many years, but that’s probably because they operated on entirely different technology platforms: Telecom on CDMA and Vodafone on GSM.
But as mobile technology has evolved, the global telecommunications industry has moved to support WCDMA technology, which has a clear path towards the upgraded 4G standard LTE.
For Vodafone, this evolution in mobile has required an upgrade to its existing network. For Telecom, it’s meant an entirely new build – at a cost of $574 million. Now both networks are using similar standards on neighbouring spectrum bands (Vodafone has 900MHz and Telecom has 850MHz), emitting radio waves from cell sites which are often located within metres of each other.
Shouldn’t both companies have foreseen there would be issues and taken steps to ensure these were mitigated?
Communication breakdown
From the affidavits filed in the Auckland High Court, it appears that before Telecom switched on the first transmitter there was a communication breakdown.
Telecom’s WCDMA investment manager Grant Crombie states that in November last year he sent Vodafone a report written by Telecom and Alcatel-Lucent “which set out Telecom’s interference requirements and also Telecom’s W850 TX emissions”. He claimed this email updated a meeting in September with his counterparts in Vodafone in which he had asked for their requirements – information which, he says, they never provided.
“Vodafone would need to tell Telecom where each site operated from on the spectrum between 900MHz and 915Mz. Only Vodafone could provide Telecom with that information, and we expected that Vodafone would do so if it foresaw, or experienced, problems once we began turning on our WCDMA sites on our W850 Network. We began the process of turning on selected WCDMA transmitter sites from November 2009. Until recently we did not hear back from Vodafone about any of these issues, and certainly were provided with no information relating to Telecom’s W8950 emissions in Vodafone’s receive band.”
Trouble at Mangakino
But it was to be four months after the first cell sites went live that Vodafone engineers began querying interference from Telecom sites. Vodafone senior radio frequency engineer Eric Liu first became aware of an issue at the Mangakino cell site on 18 March – a week after Telecom had installed its transmitters nearby.
At Liu’s request the contractor at the site asked Telecom to lock the cell site - effectively turning off the transmitter. According to Liu, when they did so, the interference disappeared.
He contacted Telecom Radio Frequency compliance manager Simon Cook-Willis and asked that the transmitter be switched off. The following week Liu learned that another two Vodafone sites co-located with Telecom’s, at Putaruru and Tangihua, “were having severe interference issues”. He contacted Cook-Willis and asked that they too be switched off. Later that month he discovered the Mangakino cell site was again experiencing interference issues, and was told by Cook-Willis that Telecom’s transmitter had “accidentally turned back on following a restart”.
By 1 April the interference issue had been escalated in Telecom, and Liu was dealing with WCDMA investment manager Grant Crombie. They both agreed to a coordinated test on Vodafone’s and Telecom’s sites at Mangakino, but results were inconclusive.
On 6 April Liu repeated requests to both Cook-Willis and Crombie that the Putaruru and Tangihua cell sites be switched off. “From our network statistics it appears that Telecom’s site at Tangihua was turned off on 7 April and the one at Putaruru on 9 April. I can see the timing as this is when the uplink quality statistics on this site returned to normal.”
Crombie states that the Mangakino site was fully audited by technicians on 8 April who checked the filter was installed correctly. They also told Crombie that the Vodafone site was fitted with a mast head amplifier. The following day the sites at Putaruru and Tangihua were fitted with filters and on 15 April Liu asked Crombie to assist with another test but – when Crombie didn’t hear anything further – on 22 April Telecom switched the sites back on.
Five days later, on 27 April, Crombie was contacted by Vodafone’s network operations manager Ian Hooker and asked to turn off the three sites. He reluctantly did so.
“I was quite clear that interference of the degree complained of was simply not achievable from our filtered transmitters. On 30 April, Eric [Liu] told me that the interference Vodafone was experiencing at Putaruru went away when Vodafone switched its mast head amplifiers out during their 16 April testing. To me, that establishes conclusively that the problem at that site – and possibly others where similar mast head amplifiers are deployed – is not due to the level of radio communication signals transmitted by Telecom.”
The Prime Minister complains
As the engineers grappled with issues at the Mangakino, Putaruru and Tangihua cell sites, complaints about Vodafone’s service were rolling in. A Mangakino policeman reported being unable to receive calls on his Vodafone cellphone – and he wasn’t alone.
In documents filed at the Auckland High Court, Vodafone claimed that during this time it received a 31% increase in customer complaints about network, a 147% increase in the number of formal complaints and a 154% increase in the number of customers calling to cancel their connections. It also claimed that it was “unable to reliably provide emergency services”.
(This last point was refuted by Telecom’s operational regulatory manager Tania Shackleton, who has responsibility for 111 calls. “From November 2008 onward, I have not noticed an increase in complaints about Telecom’s 111 service. In particular, I, and to my knowledge my staff, have not received complaints from Vodafone indicating that their customers have had trouble calling 111 services. As this would raise a clear issue of public safety, we would expect Vodafone to have contacted us about that.”)
An extensive upgrade to the Vodafone network had been completed in November 2008 and Tunncliffe – under increasing pressure from Vodafone’s senior management – was at a loss to explain the reports of poor service.
“I was told by my CEO that he had personally received complaints from other CEOs who used our service and we had had a complaint about service quality experienced by the Prime Minister and the Minister of Communications and Technology. The common inquiry was ‘what is wrong with your network?’.”
Telecom and the MED face off over NZ Communications
During this time Telecom was engaged in a ‘tit for tat’ correspondence with the Ministry of Economic Development (MED) over interference issues between Telecom’s cell sites and those belonging to New Zealand Communications [NZC has since rebranded as 2degrees]. In a letter to CEO Paul Reynolds on 17 April, the MED’s operational policy manager Wayne Wedderspoon clears the telco of any interference issues after an investigation in to co-located sites in Auckland, Wellington and Christchurch:
“We have concluded, following a series of test measurements and analysis, that Telecom’s WCDMA transmitters meet the emission limits specified on spectrum licences issued pursuant to Telecom’s management rights.”
But the letter then goes on to suggest the MED isn’t entirely convinced that Telecom is in the clear: “Despite our view, affected rightholders who take a different interpretation (i.e such as that the emissions under a spectrum licence must remain entirely within the adjacent frequency emission limit of the relevant management right) may seek an injunction under section 118 of the Act.”
And the MED also holds back from “forming a view” on whether Telecom’s transmissions comply with international radio regulations. It asks Telecom for confirmation that filters will be installed on sites in the three main cities by 29 May.
Telecom’s chief technology officer Stuart Sutton replied on behalf of Paul Reynolds, explaining that Telecom is in commercial discussions with NZC and has offered to install 45db filters to some of its WCDMA 850 transmitters – “the costs of supply and installation of the filters to be determined by arbitration between the parties”.
Sutton writes that the purpose of installing the filters is not to eliminate actual or potential interference to GSM receivers. Wedderspoon queries this point in his reply to Sutton on 21 April: “This is surprising as avoiding interference seems to be the main reason why one would install such filters.”
The letter also notes the MED is keeping the compliance investigation open and asks again for confirmation that filters to Telecom transmitters in the three main cities will be fitted by 29 May.
Sutton replies confirming that Telecom has reached an agreement with NZC over the filter installation. In response to Wedderspoon’s request, he agrees to allow NZC to see his original letter to the MED and requests to see NZC’s correspondence with the MED over the complaint.
A final email dated 24 April from Wedderspoon to Sutton restates that “we will continue to monitor cellular system compatibility issues as the new systems and services are deployed”.
IP core goes down
It’s curious that at the same time Vodafone’s engineers were raising the same concerns already being dealt with by Telecom in relation to NZC, Sutton was denying it was a network-wide issue. But his bullish tone may in part be explained by the fact that in the case of Vodafone’s interference issues, Telecom genuinely believed it was not at fault.
That belief would have been further confirmed when, on 21 April, chief transformation officer Frank Mount attended a dinner hosted by HP/EDS. Mount states in his affidavit:
“Vodafone’s CEO, Russell Stanners, told me that Vodafone’s fault resilient IP core network went down on that day and was not recovered for hours. Such a problem is likely to cause increased customer complaints such as those Vodafone has complained of in its statement of claim in this proceeding, particularly if the incident Russell Stanners told me about was not an isolated incident within Vodafone’s internal systems.”
Vodafone’s concerns mount
Meanwhile, Vodafone’s engineers had shifted their concerns to cell sites in Auckland and, with the assistance of the MED’s Radio Spectrum Management team, conducted tests on 27 April on sites in McLarens Park and Swanson.
“During the test we were able to identify that there was interference received by our SWS (Swanson site), and the interference was not from the mobile operator, NZC, which uses the frequency bands between Telecom’s and Vodafone’s 900MHz spectrum. Kevin [Everitt – MED, senior RF inspector] verbally confirmed to me and Mark [Jefferies form Vodafone] that the interference received by our Swanson site came from the nearby Telecom site.”
Later that evening Ken Tunnicliffe asked Liu to check the mast head amplifiers – the company had recently fitted 24 new-style amplifiers but decided to test a site with an ‘old style amplifier’. They tested the amplifier on a site near a Telecom transmitter which they knew had been fitted with a filter. Their tests showed that the amplifier did not produce interference.
Telecom announces launch date
By now Telecom had announced that the launch of the XT Network would be brought forward two weeks to 13 May. Tunnicliffe and his team were concerned that the interference would increase by a factor of seven to cater for an influx of customers on the XT Network and this would further downgrade their service.
“On the very day we managed to get MED’s RSM unit to undertake testing for us, Telecom announced it was bringing its commercial launch forward. It announced this despite the fact that it had known for several weeks that Vodafone was concerned about the interference it was experiencing, and I believe, knowing it could not fix its interference problem before the launch date.”
It was while testing the Swanson site that the MED told Vodafone engineers about the filter issue with NZC, which Tunnicliffe was able to confirm after speaking with NZC chief technology officer Mike Goss.
The next day – 28 April – Tunnicliffe phoned Frank Mount and Stuart Sutton, and Sutton told him that both Telecom and the MED were aware of out-of-band emissions the previous year. They spoke about the NZC deal and agreed to get their technical teams together the following day.
“I said I thought the solution was that Telecom would need to fit transmit filters to all sites and if there were any issues with Vodafone’s mast head amplifiers then we could work together to resolve those. He said that we would need to discuss who paid for the cost of the work that needed to be undertaken and I did not comment on that.”
The following day Vodafone’s GM for Corporate Affairs Tom Chignell, acting on instructions from CEO Russell Stanners, asked spectrum strategy manager Arasaratnam Sathyendran to arrange a conference call with MED senior analyst Ian Hutchings. According to Chignell’s affidavit, at this meeting Hutchings clearly stated the MED position that “Telecom’s new WCDMA transmitters were the cause of the interference and that the remedy required to resolve the issue had to be applied to the transmitting side.”
Hutchings told them that because NZC had reached a commercial deal with Telecom, the MED had not pursued the issue. He also mentioned the 2008 Alcatel Lucent report which referred to potential interference.
“Mr Hutchings advised us that the MED had considered whether an infringement notice should be served on Telecom, but had not finalised its position on this matter.”
That day Vodafone formally complained to the MED about the interference.
On 30 April at 8am, the MED’s Wayne Wedderspoon phoned Tunnicliffe to tell him he was putting out a letter to Telecom in relation to his complaint. “He told me that the real issue was, in his view, Telecom accepting responsibility for fitting filters to all its transmitters. He said that he saw this was necessary as Telecom was required to take into account the services operating in adjacent bands.”
At a teleconference between the two technical teams that evening, Tunnicliffe requested that Telecom turn off 107 transmitters by midday 1 May. “These 107 sites had been the first identified by my team as being affected, but it is now clear that the issue is more widespread than that and approximately 530 sites are estimated to be affected, although without further testing we cannot be sure that it is not more extensive than this.”
Telecom engineers argued the list be reduced to 92 sites, but gave no commitment that it would switch the transmitters off. Tunnicliffe then asked Telecom to provide them with a list by midday of the sites where the filters had been fitted, and he gained the strong impression that Telecom did not have enough filters available to install on the top 10 sites which Vodafone said were a priority.
“The Triasx filters are manufactured in Australia by a small manufacturer. They are not expensive, costing about $A1000 each, but I understand that it might take a month or two for them to manufacture enough filters to enable Telecom to fit them to all its transmitters.”
Meanwhile, Tunnicliffe had left a voicemail with Stuart Sutton emphasising that Vodafone “needed to have the issues resolved by 12pm on 1 May 2009”.
He received the following text message: “Ken, just arrived in Singapore and got your VM. Understand your point completely. We will be writing to you Friday with a formal position on your request.” It was followed by another: “Still hope that we can solve together practically, Stuart”.
Heading to court
But it was too late; the lawyers had got involved.
The next morning Tunnicliffe received another text from Stuart: “Ken, given your legal counsel has written to ours on the interference we will be formally writing back in one letter GC to GC. I have made sure that the question you asked me was covered in the letter. It is in draft and under review but you guys should get it lunchtime. Meanwhile I have instructed my guys to continue to work with your guys as we discussed.”
At 11.30am, following a telephone discussion with Telecom CEO Paul Reynolds, Vodafone CEO Russell Stanners called a meeting with his key executives.
“Mr Stanners told us that Mr Reynolds had said we would have to undertake measuring of interference signals at each of our sites to show that they were emitted outside their licence terms. Such a process would take months to undertake as there are 1069 sites to test and they are spread out across New Zealand,” says Tunnicliffe.
“A decision was made to seek an injunction to stop the interference.”
Postscript
Media statements about the court action were released by both Vodafone and Telecom at around 5pm on Friday 1 May. Over the weekend the Telecommunications Carriers Forum’s Chief Executive, Ralph Chivers, sent an email to both parties: “In the interests of our industry’s customers I am anxious that we move forward to resolve this situation as rapidly as possible. I am willing to play a role in doing so if you would find this useful.”
The offer was seized upon by Telecom, who suggested that Chivers become the statutory arbitrator in the dispute, but Vodafone rejected the idea: “The resolution of this litigation lies in Telecom’s hands, if you will provide us with an undertaking to cease the interference.”
The day before the court action, Kordia strategic technology manager Ian Goodwin carried out tests in three locations in Wellington, and established that Telecom’s transmitters were interfering with Vodafone’s receiver signals. During a visit to one of the cell sites Goodwin came upon a Chorus van installing filters on the transmitters. His findings were strongly disputed by Telecom’s principal advisor for wireless, Mansoor Shafi, in a second affidavit.
On 7 May Vodafone formally withdrew the injunction proceedings against Telecom. In a Joint Memorandum of Counsel, signed by lawyers on both sides, Vodafone sought to be relieved of damages and Telecom requested that two affidavits be sealed. There were no orders for costs.