New 3rd Generation ONT post-launch software upgrade
One retail service provider noticed some abnormal DHCP behaviour in the new 3rd Generation ONT that we have launched into production and that wasn’t noticed in the test environment. Working with the retail service provider and our suppliers, we have identified the cause and we will shortly start testing a software patch to correct the defect.
The issue is triggered by settings specific to an RSP’s network and as a result the majority of ONTs appear to be unaffected. Normal assure processes of power cycling or resetting ONT will clear abnormal DHCP behaviour so customer disruption will be minimal until the software upgrade. There are some steps that you can take to minimise impact.
What's the technical detail?
In a certain situation, where DHCP fails to renew, the 3rd Generation ONT can become non-responsive for Voice and TR-069.
In accordance with RFC2131, DHCP will attempt to renew its IP address at T1, which is typically 50% of the lease time. If the DHCP server cannot be contacted, or does not respond, due to routing or DHCP server issues at T2, which is typically 87.5% of the lease time, the ONT will use a broadcast renewal.
If the DHCP renewal is still not possible, and the lease time is reached, the ONT will, in accordance with RFC2131, go back into INIT state and restart full DHCP Discovery. However, when an IP address is eventually obtained, the defect is the G3 ONT is not inserting the routes fully, so the ONT is only contactable within its IP subnet and as a result any ACS or voice servers outside of the ONT’s subnet, cannot be contacted.
The scenario causing DHCP renew to fail is usually due to some kind of routing or connectivity issue between the ONT and DHCP server. This could be an issue in the Chorus or RSP network. The G3 ONT defect will only be triggered if the routing/connectivity issue lasts LONGER than the DHCP lease time.
Rebooting the ONT will resolve the DHCP issue.
To reduce the likelihood of situation that triggers the DHCP issue, it is recommended that the DHCP lease time setting is longer (we suggest 1 day) thereby reducing the likelihood of outages impacting DHCP renewal for both the T1 and T2 timers.
Next steps
Our ONT vendor has identified root cause, and are proposing a new software release to resolve this issue. Once we have the new software, we will then need to fully test the new software before upgrading all production 3rd Generation ONTs.
Please ensure your technical staff are aware of the issue, this is especially important for those of you who have on-boarded for NGA Voice. However all of your technical staff who look after any of the following need to be aware of defect:
- 3rd Generation ONT
- DHCP servers and infrastructure
- Voice servers
- ACS (TR069) servers.
Your Service Delivery Manager can put you in touch with technical expects if your wish to know more.
We will provide more detail of software upgrade timing once it is known.