Question 4/2: Assistance to developing countries for implementing conformance and interoperability programmes

Final Report

Preface

ITU Telecommunication Development Sector (ITU-D) study groups provide a neutral contribution-driven platform where experts from governments, industry and academia gather to produce practical tools, useful guidelines and resources to address development issues. Through the work of the ITU-D study groups, ITU-D members study and analyse specific task-oriented telecommunication/ICT questions with an aim to accelerate progress on national development priorities.

Study groups provide an opportunity for all ITU-D members to share experiences, present ideas, exchange views and achieve consensus on appropriate strategies to address telecommunication/ICT priorities. ITU-D study groups are responsible for developing reports, guidelines and recommendations based on inputs or contributions received from the membership. Information, which is gathered through surveys, contributions and case studies, is made available for easy access by the membership using content-management and web-publication tools. Their work is linked to the various ITU-D programmes and initiatives to create synergies that benefit the membership in terms of resources and expertise. Collaboration with other groups and organizations conducting work on related topics is essential.

The topics for study by the ITU-D study groups are decided every four years at the World Telecommunication Development Conferences (WTDCs), which establish work programmes and guidelines for defining telecommunication/ICT development questions and priorities for the next four years.

The scope of work for ITU-D Study Group 1 is to study “Enabling environment for the development of telecommunications/ICTs”, and of ITU-D Study Group 2 to study “ICT applications, cybersecurity, emergency telecommunications and climate-change adaptation”.

During the 2014-2017 study period ITU-D Study Group 2 was led by the Chairman, Ahmad Reza Sharafat (Islamic Republic of Iran), and Vice-Chairmen representing the six regions: Aminata Kaba-Camara (Republic of Guinea), Christopher Kemei (Republic of Kenya), Celina Delgado (Nicaragua), Nasser Al Marzouqi (United Arab Emirates), Nadir Ahmed Gaylani (Republic of the Sudan), Ke Wang (People’s Republic of China), Ananda Raj Khanal (Republic of Nepal), Evgeny Bondarenko (Russian Federation), Henadz Asipovich (Republic of Belarus), and Petko Kantchev (Republic of Bulgaria).

Final report

This final report in response to Question 4/2: “Assistance to developing countries for implementing conformance and interoperability programmes” has been developed under the leadership of its two Co-Rapporteurs: Gordon Gillerman (United States of America) and Cheikh Tidjani Oudaa (Mauritania); and four appointed Vice-Rapporteurs: Richard Anago (Burkina Faso), Lisa J. Carnahan (United States of America), Roland Yaw Kudozia (Ghana) and Osmar Machado (Brazil). They have also been assisted by ITU-D focal points and the ITU-D Study Groups Secretariat.

ISBN

978-92-61-23031-9 (Paper version)

978-92-61-23041-8 (Electronic version)

978-92-61-23051-7 (EPUB version)

978-92-61-23061-6 (Mobi version)

This report has been prepared by many experts from different administrations and companies. The mention of specific companies or products does not imply any endorsement or recommendation by ITU.

Please consider the environment before printing this report.

© ITU 2017

All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU.

Table of Contents

Preface

Final report

Executive Summary

i. Introduction

ii. Previous work

iii. Statement of the situation

1 CHAPTER 1 – Conformance and interoperability

1.1 Global trends related to conformance and interoperability

1.1.1 Harmonized standards and technical requirements

1.1.2 Acceptance of conformity assessment results and mutual recognition procedures

1.2 Issues and challenges of developing countries in the implementation of C&I framework

1.3 Approaches to meeting confidence in ICT equipment

1.3.1 Contribution of test laboratories to C&I

1.4 Progress on conformance and interoperability achieved by all ITU Sectors

1.5 Terms and definitions

2 CHAPTER 2 – Capacity building on C&I

2.1 Awareness, knowledge, training and technology transfer

2.2 Examination of effective information-sharing systems and best practices able to assist in capacity building

2.2.1 Mobilizing stakeholders to build C&I capacity

2.2.2 Assessing means and requirements regarding capacity

2.2.3 Implementing a capacity building strategy

2.3 Collaboration with testing laboratories

3 CHAPTER 3 - Implementation guidelines

3.1 Harmonization of C&I regimes to improve regional integration

3.2 Guidance on the framework and procedures to establish mutual recognition agreements

3.2.1 Identification of the scope

3.3 Innovative approaches for establishing conformance and interoperability programs

3.3.1 Smart testing procedures

3.3.2 Virtual testing

3.3.3 Certification services: MRA on regional levels and regional conformity assessment body

3.4 Market surveillance and maintenance of conformance and interoperability regimes

3.4.1 ICT equipment manufacture and transit – Multi-stakeholder environment

3.4.2 Post-market surveillance procedures

3.5 Laboratory synergy with the C&I ecosystem

4 CHAPTER 4 – C&I case studies, regional and national experiences and guidelines

4.1 Conformity and Interoperability assessment on a regional basis

4.2 C&I case study on a national experience

4.3 Case Study Library

4.4 ITU-D Guidelines

4.5 Recommendations for assessment studies on a regional basis

5 CHAPTER 5 – Roadmap for Member States for implementing C&I programs

Abbreviations and acronyms

C&I Vocabulary

Annexes

Annex 1: Conformity and Interoperability practices

Annex 2: Relevant Recommendations and Reports of the other ITU sectors

Annex 3: References

List of tables and figures

Tables

Table 1: Case studies reported in C&I events worldwide

Figures

Figure 1: Conformity, Conformance and Interoperability

Figure 2: Trends: Drastic increase of Things connected

Figure 3: Hands-on training in real testing laboratories

Figure 4: C&I assessment study in COMPTELCA region

Figure 5: MRA phases 1 and 2 for implementation

Figure 6: Remote interoperability testing

Figure 7: Remote type approval testing

Figure 8: Regional Certification Body

Figure 9: Conceptual ICT equipment trade and use under a C&I regime (mobile phone example)

Figure 10: Laboratory organization and management

Figure 11: Case studies reported in C&I events worldwide

Executive Summary

Competent conformity assessment builds consumers’ trust and regulators’ confidence in tested products and consequently strengthens business environment and, thanks to interoperability, the economy benefits from business stability, scalability and cost reduction of systems and equipment.

While economically Conformance and Interoperability (C&I) increase market opportunities, encourage trade and technology transfer, and contribute to the removal of technical barriers, C&I can enable access to ICT services.

To increase the benefits of C&I, many countries are looking forward to improve their C&I programmes both at national and bi- or multi-lateral levels. However, some developing countries C&I systems are not mature because of a number of major challenges, such as the lack of appropriate regulatory frameworks, human capacity and infrastructure. C&I can socially help spreading ICT services availability and affordability to all people at a good level of quality.

Thanks to the contributions received from the ITU-D Members, the work of Question 4/2 resulted in a Report that covers several of the issues hampering the improvement of C&I, presenting traditional solutions and offering innovative ways to approaches in developing countries.

Lessons learned and ways forward

One of the main conclusions from Question 4/2 is that collaborating on approaches that facilitate the recognition of the results of competent conformity assessments can facilitate meeting the needs of regulators, consumers and businesses in developing countries in an efficient manner. This conclusion benefits from having a specific Question in an environment that holds ITU’s core value of networking and collaboration among nations, industry and experts (here represented by the ITU-D Study Groups) allows the ICT community from different C&I backgrounds to share their views, concerns and traditional/innovative approaches to solve different aspects pertaining to this critical component of ICT networks: equipment, system, and more important, the people. These individuals range from technicians behind the scene evaluating the conformity to the final consumers, which are the main beneficiaries of all efforts directed to improve the compliance of products to technical regulations.

Considering the future scenario of Internet of Things (IoT) where billions of things are to be connected, practical application of C&I is even more critical. Countries must have C&I systems ready for this future. Standards and technical regulation targeting safety, interoperability, quality, and spectrum allocation are expected to be a key part of the agenda of decision makers. Well implemented C&I systems that are effective, efficient and minimize redundant testing will promote a smooth transition to future ICT networks.

Question 4/2 members presented differing views on the path forward for progress on the goals associated with the C&I Question study.

Question 4/2 dealt with a total of 154 documents, of which 51 were contributions received from ITU-D Members that greatly contributed to the Final Report. This inputs presented issues, challenges, efforts to improve the level of C&I, and innovative and affordable solutions to move forward. Also, seven meeting reports were generated, 12 incoming liaison statements were received, and 12 outgoing liaison statements produced. The Report presents detailed information on 13 case studies from Regional Organizations and ITU-D Members. Also, a list containing 48 country reports introduces C&I information in those countries.

There is broad agreement among all ITU-D Study Group 2 members that conformance and interoperability is a crucial issue to support providing access to ICT in developing and developed countries.

Two approaches were discussed at great length for the continuation of assistance to developing countries on implementation of C&I programmes:

– Continue with the studies under ITU-D Study Groups in synergy with BDT Programmes; and,

– Continue BDT Programmes work to support implementation of C&I in developing countries.

These approaches can:

– Continue the collaboration, research and sharing of experiences (traditional and innovative ways);

– Identify a technical requirement benchmark for standards adoption to support developing countries;

– Promote ITU-D Membership representation in other events related to C&I (e.g. WTO-TBT and ISO/CASCO);

– Benefit from the experiences of developed and developing countries that have successful implemented MRAs for C&I.

i. Introduction

Conformity assessment builds consumers’ trust and confidence in tested products and consequently strengthens business environment and, thanks to interoperability, the economy benefits from business stability, scalability and cost reduction of systems, equipment and tariffs.

While economically Conformance and Interoperability (C&I) increase market opportunities, encourage trade and technology transfer and contribute to the removal of technical barriers, they socially help spreading ICT services availability and affordability to all people at a good level of quality also supporting to achieve the Sustainable Development Goals.[1]

Inclusion of an ITU-D study group Question on C&I provides an effective way to further the aims of WTDC Resolution 47 (Rev. Dubai, 2014), WTSA Resolution 76 (Rev. Dubai, 2012) and Resolution 177 (Rev. Dubai, 2014) of the Plenipotentiary Conference.

Member States and ITU-D Sector Members can assist and guide each other by conduction studies, building tools to bridge the standardization gap, and navigating issues related to conformance and interoperability programmes.

ii. Previous work

ITU-D Study Group 2 was assigned by WTDC-14 this Question on Conformance and Interoperability for the first time for the 2014-2017 study period, so there is no previous work available to refer to.

However, taken into consideration the establishment of the ITU C&I Programme, and its Action Plan approved by Resolution 177 (2014), activities have been undertaken by ITU and Member States under the 4 Pillars of programme: Pillar 1: Conformity assessment; Pillar 2: interoperability events; Pillar 3: Human resource capacity building; and Pillar 4: Assistance in the establishment of test centres and C&I programmes in developing countries. Annex 2 of this report presents more information on ITU sectors’ relevant recommendations and reports.

iii. Statement of the situation

To increase the benefits of C&I, many countries have adopted harmonized C&I regimes both at national and bi- or multi-lateral level. However, some developing countries, have not yet done so because of a number of major challenges, such as the lack of appropriate/adequate regulatory frameworks, human capacity and infrastructure, to be in a position to test or to recognize tested ICT equipment.

In this context, the WTDC-14 Dubai Declaration[2] and the Plenipotentiary Resolution 177 (Rev.  Busan, 2014) recognized that widespread conformance and interoperability of telecommunication/ICT equipment and systems through the implementation of relevant programmes, policies and decisions can increase market opportunities and reliability and encourage global integration and trade.

During the 2014-2017 study period the Question received 154 documents. To list a few, relevant contributions informed the C&I community about: countries reports on issues and challenges to improve the conformity and the interoperability; experiences with the management of conformity assessment; innovative ways to tackle the difficulties faced related to the establishment of C&I regimes ranging from simplified testing procedures to the creation of the Virtual Laboratory services concept; and so on.

[1] WSIS 2016 Outcomes: http://www.itu.int/net4/wsis/forum/2016/Outcomes/#hlt.

[2] http://www.itu.int/en/newsroom/wtdc-14/Pages/dubai-declaration.aspx.

1 CHAPTER 1 – Conformance and interoperability

Information and Communication Technologies (ICTs) and their related services are widely recognized as key drivers for socio-economic and cultural development, as well as for regional integration. Also ICTs are enablers for the achievement of the SDGs.[1]

In a global economy characterized by rapid technological change, a variety of ICT solutions and by the convergence of telecommunication networks and services, ICT users – public entities, businesses and consumers, have understandably certain expectations regarding interoperability[2], quality and safety.

In this regard, to facilitate a safe usage of products and services anywhere in the world, regardless of who is the manufacturer or the service provider, it is crucial that products and services be developed in accordance with relevant international standards, regulations and other specifications[3], and that their compliance be demonstrated.

Conformity assessment provides confidence that ICT equipment meet international standards, increasing the probability of interoperability, defining appropriate where interoperability can be tested increases competition and reduces the chances of being locked into a single product/proprietary solution. Figure 1 shows a summary of these ideas.

Figure 1: Conformity, Conformance and Interoperability

Availability of high quality performing products will accelerate widespread deployment of the infrastructure, technologies and associated services allowing people to access to the Information Society regardless of their location or chosen device and contributing to attaining the Sustainable Development Goals.[4]

This chapter aims at describing the value of effective and efficient approaches that leverage widely accepted standards and competent conformity assessment to facilitate access to current telecommunications technology and meet the expectation of regulators, service provider and the public.

1.1 Global trends related to conformance and interoperability

The development and deployment of telecommunication equipment is happening at a rapid pace. Regulators and market forces strive to balance the need to provide access to the features and benefits associated with advancements in telecommunications equipment to the public and the need for confidence that the equipment will not harm public telecommunications systems and meet operational needs.

The expected scenario of all things connected is pushing forward more demand for C&I. Developing countries are looking for innovative ways to cope with the several issues that appears, such as: establishing common technical requirements, identifying the main technical reference at international level (standards); policies for robust C&I frameworks for promoting collaboration in a multi-stakeholder ICT environment (e.g. through the establishment of mechanisms including the acceptance of supplier’s declarations and Mutual Recognition Agreements).

Figure 2: Trends: Drastic increase of Things connected

Figure 2 presents some of the challenges in a skyrocketing increase for ICT connection. In this scenario, existing questions such as “how to achieve conformity” is taking higher priority in the agenda of public and private agents.

1.1.1 Harmonized standards and technical requirements

The use of harmonized standards and technical requirements for telecommunications equipment has value to governments, telecommunications service operators, equipment manufacturers and the public. Harmonized requirements provide greater certainty for telecom stakeholders. Manufacturers are better able to anticipate, design and manufacturer to the technical requirements that their products will have to meet. Regulators and telecom service providers have common expectations of equipment performance and quality. These lead to enhanced access to current communications technology and reliable service for the public.

For the development of international standards, guides and recommendations the TBT Committee decided on six (6) principles (November 2000, G/TBT/9):

– Transparency;

– Openness;

– Impartiality and consensus;

– Relevance and effectiveness;

– Coherence;

– Development dimension.

1.1.2 Acceptance of conformity assessment results and mutual recognition procedures

Regulators and telecommunications service providers need confidence in telecommunications equipment conformance with technical requirements. This confidence often come from understanding the competence and quality of testing and other aspects of conformity assessment. In order to facilitate a mutual understanding of competence and quality associated with conformity assessment CASCO, a joint committee of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) publish international standards related to conformity assessment activities. These standards are applied in a variety of systems and generally address conformity assessment information, quality system requirements of organizations performing conformity assessment and competence. In the telecommunications equipment sector, standards related to the use of supplier’s declaration of conformity, testing and calibration laboratories, product certification and accreditation of conformity assessment organizations are routinely used.[5]

Telecommunication equipment conformity assessment conducted in accordance with these standards can form the basis for the acceptance and/or mutual recognition of quality test results and conformity assessment carried out by competent organizations and reduce the need for duplication of testing and conformity assessment activities. The resulting confidence in the conformance of telecommunications equipment can reduce the resources needed by regulators and telecommunications service providers to manage risk associated with non-conforming products. Telecommunications equipment manufacturers are able to more effectively the process of demonstrating conformity with technical requirements in multiple markets. This can lead to enhanced access to current communications technology and reliable service for the public.

To date, ITU and ISO have an A liaison status. This A liaison status allows the ITU to participate in the activities of CASCO including membership of the relevant working groups that develop or revise the conformity assessment standards. ITU can also participate in the activities of the CASCO STAR group (Strategic Alliance and Regulatory Group - terms of reference attached). This group exists to allow industry sectors and intergovernmental agencies to participate directly in the activities of CASCO.[6]

1.2 Issues and challenges of developing countries in the implementation of C&I framework

C&I issues are driven by a variety of concerns and problems, including but not limited to:

– Legacy Intelligent Network signalling (Interoperability problems) services behaviour when replacing equipment, signalling in mobile network (e.g. access, core, SMS);

– Lack of conformity and interoperability between equipment from different vendors;

– Equipment from different manufacturers, due to application of non-standardized interfaces or protocols;

– Equipment from one manufacturer, but with different software revisions resulting in incompatible SIP[7] clients;

– Conformity of STB equipment of different IPTV middleware manufacturers;

– Bandwidth: Capacity of transmission of voice, data and video when users add much to the existing network;

– Complexity of networks to achieve interoperability through integration of networks and devices;

– Services launched with some providers not providing infrastructure and support teams to enable interoperability with other operators;

– Defining the methodology for adopting standards;

– The management of the CDR for billing;

– Implementation of new features and services on all platforms;

– Existence of different charging model;

– New technology not able to interoperate with legacy equipment;

– No testing centres and facilities;

– Lack of trained personnel to undertake C&I tasks;

– ISDN support problems;

– User terminal problems both different systems;

– Interoperability issues between services and terminal equipment that customers use ;

– Proprietary and non-standard interfaces of Vendors.

Many developing countries are now adopting more robust market access controls for the Telecommunications/ ICT equipment and systems deployed in their countries.

Even though there is a need for conformance and interoperability framework in the developing countries, there are a number of challenges that hinder the implementation. Some of these challenges are highlighted below.

– Costs:

• Mandatory in-country testing, rather than accepting test results from suppliers or other countries is cost prohibitive for developing countries both in terms of CAPEX and OPEX;

• Preparation of documentation for submission to the regulator involves cost;

• Testing to meet specific national standards not aligned with international standards requires additional cost.

– Lack of human capacity and of training opportunities: The regulator or any other agency who establishes test labs requires trained human resources. The cost involved in the salaries and other benefits and regular training is normally not feasible for developing countries.

– Weak institutional systems :

• Standardization;

• Testing;

• Certification;

• Market surveillance; and

• Lack of legal provisions and mandate on an institutional basis.

– Delays:

• Time taken to approve a product after documents are submitted;

• Testing to national standards not aligned with international standards.

– Awareness on the Standardization: Many developing countries are not aware of the need for standardization.

– Interoperability challenges manifest in different dimensions and more pronounced when new applications and services have been used and promoted by different entities such as in the implementation of e-Health and e-Government programs. As ICT services and applications find place in all kind of human life, and the proliferation of IoT becomes a reality, conformance and interoperability will pose a serious problem for developing countries if not prepared in time.

1.3 Approaches to meeting confidence in ICT equipment

The confidence in ICT equipment for regulators, users and the market with respect to ITU Recommendations, standards and technical requirements can be met through a variety of conformity assessment approaches. These approaches include the use of supplier’s declarations of conformity, acceptance of test reports from competent laboratories and certification. The rigor and independence of acceptable conformity assessment should be aligned with the risks associated with the equipment’s non-conformity with ITU Recommendations, standards and technical requirements. Recognizing that telecommunications equipment is required to meet similar technical requirements in many markets it follows that conformity assessment approaches should be organized to reduce redundant conformity assessment activities that may not add value to regulators’, users’ or consumers’ confidence. The development of approaches that maximize the acceptance of conformity assessment can be facilitated by enhancing the understanding of countries, regions and sub-regions technical requirements including their differences, perception of risk associated with equipment non-conformity and information needs.

1.3.1 Contribution of test laboratories to C&I

Value of testing

The most straightforward way to estimate the value of testing is by calculating the losses avoided, in terms of money, time and human life. Risk analysis is made in order to define the most appropriate testing strategy, by evaluating the impact of possible malfunctions with reference to the most important requirements. The evaluation may be qualitative, but whenever possible, it is reasonable to give also a quantitative evaluation, and balance any anomalies that have been detected (in terms of prevented damage or loss) against the benefits of testing, in addition to considering the cost of testing activities (in terms of resources). Further analysis would consider the impact of any future damages and costs related to the moment when the anomaly would be detected in operation.

Acquired knowledge

A further value of testing is given by the know-how obtained during the execution of testing activities. The expertise is related to the analysis of the requirements translated into a real solution, configuration aspects referred to services, equipment and environment, the knowledge of interfaces and related protocols and of the expected behavior. For a telecommunication operator, it is often an opportunity for the personnel having a direct technical and physical contact with pieces of equipment in a testing environment.

Integrated vision

The orientation of testing activities to efficiently simulate all functionalities of ICT networks, (equipment, services, interworking, external environment, etc.) means that testing is often the first time that all the different elements composing the chain are put together. It is therefore in the testing environment design phase that it is possible and necessary to deepen the detailed analysis and architectural combinations definition, and identify critical situations.

Critical view elements in a ‘mature’ testing area

Testing in a mature situation according to international standards enables specific experience of the tester role, identifying risks and preventing anomalies that could seriously impact safe deployment in real operation conditions.

This allows an up-stream evaluation of the reliability of providers, identifying products that could generate unacceptable costs and delays. This could also help providers to refine their forecasts on availability of products/services, on the basis of previous experience and statistics.

Test environment – troubleshooting

Building such a test environment will allow subsequent troubleshooting activities to identify failures or defective functioning during critical situations at implementation that were unforeseen in the initial requirements. Due to the time needed to build and study a troubleshooting environment, only a working test plant already prepared during previous phases is compatible with a cost benefit balance.

Historical documentation

The documentation of test results may be of high value for customers. For instance, helping operator purchasing departments in the choice and negotiation with providers, identifying weak points (e.g. interoperability faults), and highlighting the probability of delays in provision. It can also be of guidance to an efficient incoming quality verification process.

In a regulated market, the test results can be used by surveillance or investigative activities by contributing relevant information for the recording of overall conformance assessments.

In summary, some contributions of test laboratories[8] can be highlighted:

– Ensure that products commercialized or used in the country meet the minimum requirements (quality, safety, spectrum allocation, interoperability, etc.);

– Enhance user safety and quality of services and products;

– Increases product quality and reliability as well as users satisfaction;

– Provides human knowledge acquisition (on ICT technologies, testing methodologies, equipment configuration);

– Contributes to enhance capacity and knowledge exchange with government agencies, universities and R&D centres.

1.4 Progress on conformance and interoperability achieved by all ITU Sectors

Conformance and interoperability issues have been addressed throughout the ITU. The main body of this report addresses the activities of the ITU Telecommunications Development Sector. Annex 2 summarizes the most significant activities of the other two sectors of ITU.

1.5 Terms and definitions

For easy of reference, the C&I Vocabulary with the terms and definitions used throughout this report are located after Chapter 5.

2 CHAPTER 2 – Capacity building on C&I

2.1 Awareness, knowledge, training and technology transfer

Raising awareness of problems related to the lack of conformity and interoperability and potential harm to the evolution of ICT infrastructure that it may cause in the future would be the first step.

Given the major challenges that exist in terms of implementing structural projects for the development of information and communication technologies, the developing countries continue to encounter difficulties, in particular with regard to the human factor and lack of expertise.

Capacity building in the area of conformance and interoperability (C&I) of telecommunication equipment is the process by which developing countries build their capacity to address the risks associated with poor-quality equipment and with problems affecting interoperability of telecommunication equipment.

The essential capacities that can be strengthened in order to carry out the main functions associated with the implementation of conformance and interoperability of telecommunication equipment are:

– Human capacity: the professionals consulted by governments on issues relating to conformance and interoperability of telecommunication/ICT equipment. A country that lacks such professionals in the C&I field is obviously at a disadvantage when it comes to assessing the risks associated with poor-quality equipment and with problems affecting interoperability of telecommunication equipment.

– Institutional capacity: defined as the institutions on which governments rely for assessing conformance and interoperability of telecommunication/ICT equipment, including test laboratories, national authorities responsible for telecommunication/ICT standards, and so on.

2.2 Examination of effective information-sharing systems and best practices able to assist in capacity building

This section contains suggestions regarding effective systems of exchange and good practices for implementing the process of capacity building in developing countries in the field of conformance and interoperability of telecommunication/ICT equipment.

2.2.1 Mobilizing stakeholders to build C&I capacity

An initiative to build C&I capacity may involve the following stakeholders:

– Telecommunication regulatory agencies;

– National telecommunications standards authorities;

– International standards bodies;

– Testing laboratories;

– Conformity assessment bodies;

– The media.

2.2.2 Assessing means and requirements regarding capacity

In the context of capacity building in the field of C&I, assessment of capacity involves the following:

– Identification of priority C&I issues to be addressed in this context. A proposed Questionnaire is presented by Annex 1.

– Identification of (i) existing strengths and (ii) gaps, weaknesses and challenges.

2.2.3 Implementing a capacity building strategy

Developing countries can mobilize the following means, at national and regional levels, in order to implement a capacity building strategy in the field of C&I:

– Enhanced cooperation between institutions responsible for assessing conformance and interoperability of telecommunication equipment with a view to sharing best practices.

– Academic training on assessing conformance and interoperability of telecommunication equipment in partnership with equipment manufacturers.

– Coordination of regional events on the assessment of conformance and interoperability of telecommunication equipment.

– Active participation in international technical committees in the C&I field.

2.3 Collaboration with testing laboratories

– Partnering with real testing laboratories has been proved a good option as hands-on practices help students to perceived what is about to implement and, more importantly, to maintain an accredited test laboratory fully operation.

– Different C&I test domains can be considered for an on-the-job experience, for instance:

• Electromagnetic compatibility (spurious emissions, resistibility);

• Mobile networks (3G, 4G, 5G);

• Next-Generation Networks (protocols: SIP, SIGTRAN, MEGACO, etc.);

• Batteries (Lithium batteries, for telecommunication stations);

• DTV receivers;

• Cables;

• Specific Absorption Rate (SAR);

• Electrical safety.

Figure 3: Hands-on training in real testing laboratories

3 CHAPTER 3 - Implementation guidelines

3.1 Harmonization of C&I regimes to improve regional integration

There are many activities that will promote harmonization and integration. Some of these activities can be undertaken by members including:

– Enhancing participation in the development of international standards catalyzing technical solution in standards that meet the their needs and developing an enhanced understanding of the application of international standards to telecommunications equipment;

– Enhancing members understanding of international standards for conformity assessment activities such as those developed in ISO/IEC CASO;

– Participating in international accreditation cooperation such as the International Laboratory Accreditation Cooperation (ILAC) and the International Accreditation Forum (IAF) and regional cooperation such as the Southern African Development Community in Accreditation (SADCA) and the Inter-American Accreditation Cooperation (IAAC), the Arab Accreditation Cooperation and the Asia Pacific Laboratory Accreditation Program ; and

– Participating in the IEC System of Conformity Assessment for Electrotechnical Equipment and Components (IECEE).

These activities may provide a foundation for the implementation of harmonized standards and technical requirements and methods to accept the results of quality conformity assessments.

In the framework of the Conformity & Interoperability Programme, conformity and interoperability assessment studies are being conducted in the Regions. These studies aim to identify all the necessary elements to promote collaboration among regional and sub-regional organizations for establishing a common Conformity & Interoperability Regime through Mutual Recognition Agreements (MRA) and/or regional test centres, as appropriated. Possible scenarios to meet Member States and Regions needs on Conformity & Interoperability are presented. Regional (e.g. COMTELCA Assessment Study in 2015).[9]

3.2 Guidance on the framework and procedures to establish mutual recognition agreements

Effective implementation of mutual recognition agreements is generally accomplished in steps. The first step is the exchange of information related to the standards, technical requirements and regulation that are in place or in development in the participating countries and markets. This information exchange is followed by confidence building steps to enhance the mutual understanding of the application of standards, technical requirements and regulations including conformity assessment procedures. Confidence building measures also allow for demonstrations that conformity assessment activities to applicable standards, technical requirements and regulations can be conducted with the expected levels of quality and competence. These steps create a foundation that facilitates the acceptance of the results of conformity assessment conducted in the framework of mutual recognition.

Figure 4: C&I assessment study in COMPTELCA region

Source: ITU, COMTELCA

3.2.1 Identification of the scope

Each country interested in establishing an MRA has to assess and determine the set of its technical requirements, including technical regulations, standards and specifications. Each party will agree to accept the conformity assessment results prepared by conformity assessment bodies from the other party of telecommunication equipment meeting its set of technical requirements. Both parties have to agree to these sets of technical requirements which may need adjustments if there are disagreements.

The two main conformity assessment procedures required by regulatory authorities or manufacturers are:

– Test reports for certification or self- declaration;

– Certification.

The MRA on conformity assessment addresses the above procedures by dividing them into phases:

– Phase 1 – Mutual recognition of testing laboratories and mutual acceptance of test reports prepared by the testing laboratories.

– Phase 2 – Mutual recognition of certification bodies and mutual acceptance of certification prepared by the certification bodies.

Figure 5: MRA phases 1 and 2 for implementation

There are some MRA frameworks available for consultation, as the APEC[10] and CITEL[11].

3.3 Innovative approaches for establishing conformance and interoperability programs

During the Study Period of Question 4/2 ITU-D Members have presented innovative approaches for dealing with the most critical scenarios for checking the conformity of ICT equipment, such as tagging procedures for a massive usage of devices during sportive events (e.g. FIFA World Cup 2014 and the Rio Olympics 2016) and the potentials of virtual testing services. The reported procedures have in common the purpose of bringing effectiveness and cost reduction.

3.3.1 Smart testing procedures

The experiences shared from previous massive sportive events[12] demonstrated the direct effect from a drastic increase of telecommunication usage as well as heavy demands for accessing a National Radio Spectrum resource. This scenario requires proper planning and coordination among different actors in order to secure safety and quality of the telecommunications services provided to the population.

All stakeholders, including the organization bodies, media coverage, infrastructure providers, and users need to be aware of the regulations in force, especially the rules governing the use of ICT equipment in place.

Therefore there is a need to adopt some simple procedures during the events to verify if an equipment are compliant with regulations at national and local levels. Such procedures could be applied when there is lack of conventional testing facilities using ordinary spectrum monitoring instruments.

Procedures involves testing and labelling[13] are described in details by the Annex 1, Section 1.2 of this Report.

3.3.2 Virtual testing

The concept of virtualization of services through the Internet is a trend in the ICT sector. This new reality is also true for emerging mechanisms for assessing the connectivity of ICT equipment over IP networks and is aligned with the requirements of new converged networks.

Seeking timing, affordability, and sustainability of testing services, virtual laboratories can be an option to developing countries where there no testing capabilities.

Two solutions are here presented[14] for virtual testing: Remote interoperability testing, and remote type approval testing.

Remote interoperability testing

Objective: Assess the network of operators in different countries/regions – interoperability testing.

Worldwide experience has consolidated the need for testing and standardized certification procedures on products and systems based on Information and Communication Technology (ICT) due to the numerous problems when they are performed causes the user and the operator.

Figure 6: Remote interoperability testing

The most common problems are:

– Reduction of communication rate;

– Low reliability in communication;

– Short life useful and lower than the recommended for devices and equipment;

– High-energy consumption;

– Interference of a service in another (especially in wireless systems);

– Bad equipment purchase that do not allow evolution and compatibility with new technologies and protocols;

– Lack of interoperability of some equipment with others causing bottlenecks in communication and often very difficult to diagnose;

– Network performance fluctuations due procedures lack in the monitoring changes in equipment and software;

– Difficulties in the interconnection of different equipment manufacturers and between networks of different countries.

The laboratory performs testing for: product development, Regulatory Authority certification, pre-conformance and interoperability testing for all ICT products, compliance evaluation for mobile devices and IP protocols, and field service.

Target Audience: telecom operators, equipment manufacturers and user experience (multiple interests – customers, operators, associations, regulators, etc.).

A close and strong partnership with major equipment manufacturers of test and measurement system is desirable, which guarantees a very fast update of its infrastructure.

Remote type approval testing

Objective: Access to physical infrastructure through remote access platforms

Allows laboratory development, pre-conformance, conformance and interoperability testing in samples of ICT products by remote or virtual mode using the infrastructure of the CPqD. The samples will be supplied by entities (community involved).

Figure 7: Remote type approval testing

The services offer by Lab can be done in phases:

Phase 1: Remote training, for example, focusing on objectives and technical aspects of the main tests to be performed. The involved group will define the complete scope.

Phase 2: Conducting tests on the samples sent by the community involved to the Laboratory (scope according to the proposed objectives for each project) with video transmission of each step and sending data to reports composition. These results will be compared to the expected results in accordance with the applicable standards.

Phase 3: Testing promoted by Laboratory in home location for some types of products, often focusing on core network products (heavier and denser in infrastructure needs).

Phase 4: Providing infrastructure for conducting remotely testing (required investments in suitable test measurement infrastructure).

Phase 5: Consulting and training for providing local infrastructure for testing (if there is interest).

Requirements: Applicable standards, testing screening, etc.

– Compliance with relevant international standards. The laboratory can present an overview of the applicable international standards and experiences of operators so that the community can choose the list of references to be adopted in the trials. The architecture of each network, including aspects of future planned core network, is essential for better targeting of the results

3.3.3 Certification services: MRA on regional levels and regional conformity assessment body

From the Central American region,[15] discussions on a possible MRA lead to an innovative approach for assessing the conformity related to type approval analyses: the establishment of a Regional Certification Body.[16] Having in mind that decisions on the way forward in that region are ongoing, the concepts of this innovative approach are quite appealing to share and therefore are presented as follows:

Figure 8: Regional Certification Body

The Regional Certification Body (RCB) pertains to the Regional Organization in charge of Telecommunications in which it is transferred the responsibility to perform the conformity assessment and to issue the Certificate of conformity that must be accepted by participant countries of this agreement.

3.4 Market surveillance and maintenance of conformance and interoperability regimes

Market surveillance practices are considered a relevant tool, adopted by several developed countries,[17] to promote the conformity and interoperability of ICT equipment and the spread of consistent and reliable broadband infrastructure. Through this contribution we present some of the vast list of implementable activities by the regulatory authority and by other relevant stakeholders in order to guarantee an orderly market place for telecommunications/ICT equipment.

3.4.1 ICT equipment manufacture and transit – Multi-stakeholder environment

A typical ICT equipment manufactured and traded scenario under a C&I regime is shown in Figure 9. As one can see, several stakeholders are involved in the process. To list a few: operators, mobile phone dealers/authorized agents, environmental agencies (disposal of user equipment), revenue agency (border control), traders associations, bureau of standards, consumer defense groups, ICT regulatory agency.

Figure 9: Conceptual ICT equipment trade and use under a C&I regime (mobile phone example)

Source: Brazil, SG2RGQ/149, “Post market surveillance: practices, figures and fact”, Federative Republic of Brazil..

Different actors within their different capacities work to prevent the access of unqualified equipment in the market. ICT tools, such as electronic databases available at checkpoints and/or during enforcement activities, are quite important to responsiveness. From the example pictured above, the mobile market has different applications to prevent the use of unauthorized devices, such as the one adopted by Anatel (SIGA).[18]

3.4.2 Post-market surveillance procedures[19]

Goal:

To verify that a legally marketed product maintains its compliance with the requirements based on its approval and that the same technical characteristics and regulations required in the initial approval are maintained.

Definitions:

Technical evaluation: A set of activities to compare the relevant regulation with the factual situation or product (e.g. inspection, visual assessment, parametric measurements and conformity tests).

Sample: is the number of product units required for the technical evaluation.

Collection: the act of collecting the products approved in the market, with the manufacturer or the supplier (the product must be sealed).

Demand for inspection: request of the inspection activity.

Regional unit: Regulatory Authority unis located in a different states or region.

Collection Term: form used by the Inspector to document the collection.

CHS: IT system that manages the certification and homologation database.

Methodology and general procedures

1. Planning of the inspection action (before going to the field)

Collection of information

– Search information in the system (CHS);

– Search product information for the name of the model, the manufacturer or the holder of the approval (the inspector must have access to all product information in the system);

– Check the product’s status (e.g. if it is approved, suspended, etc.).

Number of samples

– Based on the product availability;

– Where the inspection site is not defined, the supervisor may use the information contained in the system (CHS) regarding the manufacturer or the representative of the product;

– The points of sale may be selected through online search;

– Inspection may be carried out at storage sites, production line, transportation chain, exhibitions or during the commercialization of the product, preferably in the retail market.

Preparation of the Collection Term

– Evaluate the information necessary to prepare the Collection Term for the collection of products from the market (e.g. location, manufacturer, type, model and quantity).

– The Collection Term must be filled out by the time the action is taking place.

– Based on the information obtained in the previous stages, the Inspector in charge must prepare in advance the necessary documents, as well as the logistics for the execution of the inspection action.

– If the distribution and manufacture of an approved product are performed in inspection areas of two or more Regional Units, the Inspection Department from the Regulatory Authority must be communicated so that, if necessary, it can centralize the planning and conclusion of the oversight action.

2. Selection of the sample

For the selection of the sample, the Inspector should observe the following criteria:

1) When not indicated otherwise in the demand for inspection, the sample to be selected must be a complete piece, that is to say, it must be taken in the same way as it is supplied to the consumer, including its packaging, when applicable.

2) In selecting the cable sample, the Inspector must search for a minimum size containing all markings, including the approval code issued by the Regulatory Authority.

3) The selection of the sample at the collection site should be done by the Inspector at random from among the available products.

The sample to be selected must be the one required for the performance of tests and/or technical evaluation.

3. Collection

The Inspector should observe the following procedures for sample collection:

1) The quantity of products should be observed in accordance with the action plan;

2) The Inspector must ensure the traceability and integrity of the sample collected;

3) The selected sample must be properly identified.

When the product is collected in the retail market, the Inspector shall additionally:

1) Verify the traceability of the product supply, ensuring that the homologation applicant is part of the distribution chain. Traceability may be evidenced by means of fiscal document (s) and / or Traceability Statement (s);

2) Inform the responsible for the retail establishment that Regulatory Authority will determine to the supplier the replacement of the product, and that the costs resulting from this substitution will be borne by the holder of the Certificate of Homologation of the product, which will be notified by means of an official letter.

4. Technical evaluation

Visual inspection:

– The Inspector must perform a visual inspection on the sample collected. To this end, the Inspector must take into account the information recorded in the application for approval.

– In carrying out the visual inspection, the supervisor should consider the following items:

1) Identification of the product model, observing that it complies with the model described in the Certificate of Homologation. The Inspector must take into account that the model described in the type-approval certificate is not always the commercial name of the product. Thus, the Inspector can check the existence of a label or mark on the product indicating the model. For small size products, model evaluation can be done by comparing images (e.g. cabinet, board, components, among others).

2) The Inspector should note the serial number of the product and list the existing accessories (e.g. suppliers, peripherals, headset, etc.).

3) In the evaluation of power suppliers, when it is possible for them to be identified, the Inspector should check that the source supplied is the same as the ones used in the electromagnetic compatibility tests carried out in the type approval stage. This data can be obtained from the most recent Conformity Assessment Report within the homologation process. In the case of a mobile phone that is accompanied by a charger, the Inspector shall observe whether the charger is listed in the Certificate of Conformity issued by the Certification Body.

4) Identify the manufacturer, the country of origin of the product whenever possible, and compare it with the data of the type approval certificate and the Certificate of Conformity, and verify if it has common characteristics;

5) Ensure that the item collected has all the components, accessories and manuals, packed as it has to be supplied to the final consumer. For products subject to Specific Absorption Rate (SAR) tests, observe whether the accessories that have been tested are the same as those accompanying the product in its packaging;

6) Identify specific conditions that may exist related to the type approval of the equipment under examination (e.g. seal, approval number, declarations);

7) Observe position of buttons, openings, and connections in order to validate its similarity to the product certified, according to the photos available in the homologation database;

8) For cables subject to approval, the Inspector may use the internal photos contained in the test reports to evaluate the conductors, shielding, coating, etc.

9) Evaluate the functionalities in the collected product, comparing those that can be homologated and which are described in the homologation certificate;

10) Verification of internal modules, circuit boards, arrangement of components and internal components, by comparing the internal photos contained in the approval request. In case the opening of the product could compromise the structure of the equipment, the person responsible for the product should be contacted to support this activity; and

11) Other visual evaluations that is necessary to meet the demand for inspection.

– The results of the evaluations should be included in a Report highlighting the compliance achieved with the Agency's regulations or determinations.

Conducting tests on the product:

– When necessary or determined in the demand for inspection, the measurements and tests will be carried out by test laboratories recognized by the Regulatory Authority or by the inspector himself, when he has the instrumental and technical capacity, adopting the procedures established by the technical requirements.

– When the tests are carried out by a laboratory:

1) The Inspector shall indicate the necessary tests to demonstrate the maintenance of technical compliance with the established requirements.

2) The inspection should adopt the necessary procedures to transfer the sample to laboratory tests, guaranteeing its integrity and traceability.

3) The test laboratory shall issue a report containing the results of the tests without a conclusive opinion or any judgment that may influence the analysis of the results.

– When the tests are carried out under the Inspector auspices, the principles described in the current procedures must be observed within the test procedure to be adopted and the requirements for its accomplishment.

5. Evaluation of the results obtained

– Upon receipt of the test report or the tests carried out under these procedures, the Inspector shall evaluate the results in accordance with the current technical requirements.

– The results of the evaluations should be included in a Report or Inspection Report highlighting the compliance with the Agency’s regulations, or non-compliance, accordingly.

6. Completion of the supervision action

– After the technical evaluation and preparation of the inspection report, evidencing regulatory noncompliance to the determinations of the Regulatory Authority, a process must be instituted against the offender.

– The Regional Unit will decide on the appropriate destination for the sample collected, considering the various stages of the process, notifying the supplier or the manufacturer.

3.5 Laboratory synergy with the C&I ecosystem

A typical laboratory organization and management structure[20], as well as the roles and responsibilities, in which human resources profiles and skills should be pursued, is presented in this section.

Laboratory organization and management

An example of a laboratory organization and management can be summarized by the following figure with well-defined attributions within the organization.

Figure 10: Laboratory organization and management

– Director with overall responsibility for the management and operation of the test laboratory.

– Technical Manager to manage, develop and maintain human, capital resources, and infrastructure.

– Quality Manager to establish quality policies, make sure day-to-day lab activities comply with ISO/IEC 17025 requirements, lead internal audits, regularly report on the effectiveness of the quality management system to top management, facilitate corrective actions, identify new accreditation needs and organize inter-laboratory tests.

– Innovation and Marketing Manager to map market and competitors, identify new opportunities and customer needs, and analyze/indicate investment priorities.

– Sales Manager to manage the sales team and define cost and market-based pricing.

– Logistics Coordinator to coordinate the logistics team – sample receipt and tracking within testing center.

– Laboratory Coordinator to coordinate (schedule and work shifts) and supervise testing activities performed by his/her team, in compliance with quality standards and meeting customer requirements.

– Testing Specialist to develop and implement new testing services in compliance with quality standards, and provide training.

– Testing Operator to execute the test campaign activity.

Human resources – Profiles and skills

– Technical Manager: People management, activity planning and monitoring, business plan development, cost and budget planning, negotiation skills, understanding of business trends (market, regulatory and political), equipment/technology expertise, knowledge of quality standards and basic knowledge of foreign languages.

– Laboratory Coordinator: Strong ability to coordinate teams and monitor activities, knowledge of quality standards, test methodology, test report analysis, elaboration of technical proposals, management reports, workflows, knowledge of Good Laboratory Practice (GLP), technology expertise.

– Testing Specialist: Strong ability to train people, high level of technology/equipment expertise, excellent operational skills, use of statistical techniques, test methodology development, interpretation of test standards.

– Testing Operator: Test methodology, elaboration of test reports, interpretation of test standards, knowledge of Good Laboratory Practice (GLP), technology/equipment expertise, operational skills, basic knowledge of quality standards and statistical techniques.

Laboratory quality management system

A laboratory quality management system is a framework of policies and procedures that laboratories operate under. Some of the key issues that an effective quality management system must address are:

– Factors contributing to the reliability of results and uncertainty;

– Human factors;

– Accommodation and environmental condition;

– Test, calibration and validation methods;

– Measurement traceability;

– Sampling;

– Handling of test and calibration items;

– Quality control procedures;

– Competence requirements for key personnel;

– Demonstrating competence of key personnel;

– Inter-laboratory comparison of test results.

To meet the requirements of ISO/IEC 17025, the laboratory shall establish and maintain procedures for:

– Management system;

– Document control;

– Identification, collection, indexing, access, filing, storage, maintenance and disposal of quality and technical records;

– Selection and purchasing of services and supplies;

– Implementing corrective actions when nonconformity is identified.

The laboratory shall:

– Continually improve its management system and operations;

– Periodically conduct internal audits of its activities;

– Manage its impartiality including relationships with other organizations; and,

– Ensure the protection of its customers’ confidential information and proprietary rights (including protection of electronic storage and results transmission).

Challenges

Rapid development of technologies have direct implications on:

– Maintaining laboratories technical infrastructure;

– Maintaining current standards and technical requirements; and

– Maintaining technical competence.

4 CHAPTER 4 – C&I case studies, regional and national experiences and guidelines

4.1 Conformity and Interoperability assessment on a regional basis

In the framework of collaboration among regional and sub-regional organizations for establishing a common Conformity and Interoperability (C&I) regime and mutual recognition agreements, the International Telecommunication Union (ITU) is carrying out conformity and interoperability assessment on a regional basis.

Maghreb region

The objective was to carry out the Conformity and Interoperability Assessment on a regional basis. This project aimed to identify all the necessary elements and promote the Collaboration among Regional and Sub-regional Organizations for establishing a common C&I Programme and Mutual Recognition Agreements (MRAs). It also presents possible scenarios to meet the needs and interests of Member States and Regions on C&I. The Assessment covers, among other items:

– General aspects of Maghreb region;

– Regulatory framework and institutions;

– Accreditation, laboratories, certification bodies and marking;

– Recommendations for establishing common C&I programme and MRAs.

Outcome of an assessment study:

– Develop a plan for the establishment of Regional C&I Testing Laboratories. This plan shall include the criteria to determine the numbers, locations and testing scopes of these Regional C&I Testing Laboratories;

– Develop a plan for the establishment of a Maghreb MRA framework similar to the MRA already signed between countries in other regions (ex: Asia Pacific Economic Cooperation Mutual Recognition Arrangement for Conformity Assessment of Telecommunications Equipment (APEC TEL MRA)) covering 21 economies in the Asia Pacific region. This plan shall preferably include the creation of a small Maghreb MRA Task Force which can adapt the MRA models already used, to meet specific requirements of the Maghreb Region. In implementing the Maghreb MRA, Members States shall become signatories to this multilateral arrangement in the Maghreb Region Participation in the Maghreb MRA is voluntary.

Caribbean Telecommunication Union

Many English-speaking Caribbean[21] jurisdictions are aggressively promoting entrepreneurship and innovation, with the development of ICT products and services as a key focal point. The limited size of the very important development communities has challenged the viability of local and even regional testing and certification centres yet the absence of such facilities remains a barrier to further development of indigenous ICT-based innovation for both hardware and software products. In addition to the unsatisfied need for testing of indigenous products which comprise communications components, is the need by operators and regulators in the telecommunications and more broadly ICT sectors to ensure the conformity and interoperability of equipment utilized in public networks.

The need on the part of the local and regional innovation communities as well as that of telecommunications regulators and service providers, motivates an examination into the feasibility of establishing regional Conformance and Interoperability facilities. However Small Island Developing States (SIDS), as found in the Caribbean Region, face unique challenges that are not captured by the World Bank’s (<US$ 11,905) GNI per capita per year classification of developing countries. These include the crippling lack of economics of scale, and limited human, institutional, and financial capacities; as well as considerable vulnerability to disruption by natural hazards and economic events.

Mindful of the risks inherent in the establishment of facilities which require substantial capital and operational expense; and also mindful that shortcomings in critical mass, such as obtain in SIDS, exacerbate these risks, three activities are proposed in a manner that is cognizant of both demand and supply issues, responsive to existing needs across disciplinary boundaries; grounded in existing institutional mandates and that seeks to determine the need for national or regional testing facilities to support C&I regimes in the Caribbean and define a way forward that is economical, efficient, effective and impacting. The specific activities proposed are:

– Activity A: Demand Study: Caribbean Conformance and Interoperability needs;

– Activity B: Risk Assessment: Cost-Benefit analysis for C&I Regime implementation;

– Activity C: Feasibility Study: National, Regional and Distributed Models of C&I testing.

These activities will require the input of several agencies namely:

– National/Regional Technical Advisors (e.g. Universities);

– National/Regional Telecommunications Regulators;

– National/Regional Standards Bodies;

– National/Regional Telecommunications/Trade Agencies;

– National/Regional/Extra-Regional Commercial Testing Labs;

– National/Regional Electronic Product Developer(s);

– National/Regional Electronic Product Importers/Vendors (esp. Telecoms Equipment);

– Potential host countries for national/regional labs;

– National/Regional Health/Environmental policy bodies.

COMTELCA

In the framework of the ITU Conformity and Interoperability Programme, conformity and interoperability assessment study has been conducted in the Central America Region in 2015.

The study identified critical elements necessary to promote collaboration among regional and sub-regional organizations for establishing a common Conformity & Interoperability Regime. The final report is available online (http://itu.int/go/M5DO), and it was presented during the C&I Validation Workshop for COMTELCA member countries, from 7-9 December 2015, in Tegucigalpa, Honduras (http://itu.int/go/5MKS).

Through the COMTELCA Standardization Committee, the follow-up of the results are being considered. In summary, the key issues consist of: strengthen the capacity of normalization in the region, the establishment of Mutual Recognition Agreements (MRA) Model and share of regional capacities for development testing.

The technical activities are being development under the COMTELCA Standardization Committee. The following meeting to move forward the implementation of proposed actions took place in December 2016, in San Salvador/El Salvador, covering the following aspects:

– Harmonization of standards (e.g. spectrum allocation, mobile, etc.);

– Mutual Recognition Agreements;

– Virtual testing platforms, in collaboration with Research Centers (CPqD has been considered as a possible partner on this activities);

– Identification of capacity building needs in terms of C&I.

4.2 C&I case study on a national experience

Brazil

As in other countries, Brazil[22] is concerned about the use of non-compliant products in telecommunications networks, which have affected the quality of service delivery.

In the 1990s, the Brazilian government opted to control the telecommunications sector, in addition to other sectors of the economy, through regulatory agencies. Since then, the National Telecommunications Agency - Anatel, created in 1997, is responsible for regulating the certification of telecommunication / ICT products and overseeing the internal market.

The current national C & I regime adopted has been established in 2000 and time allowed the maturation of the practices concerned. In view of the needs for developing countries to implement C & I regimes to combat non-compliant products. The Brazilian case study presents some aspects concerning the treatment of compliance and interoperability in its territory.

The current model current model[23] for accreditation, established in 2000, with the publishing of Resolution No. 242 which lays down general rules and procedures for the certification and approval of ICT equipment. The framework consists of an independent certification body, and 3rd party laboratories run by the private sector.

The certification process and the homologation of telecommunications / ICT products in Brazil involves, in addition to Anatel, laboratories and certification bodies which are private entities and monitored by the agency.

In Brazil, telecommunications / ICT products are divided into three categories which impose different obligations to be fulfilled.

The authority of Cameroon through its contribution to Q4/2[24], warns of the need for training to raise the awareness of stakeholders about the importance of compliance testing and interoperability

Brazil[25] also highlights some political drivers for the adoption of market surveillance procedures in that country, pointing out the following general objectives:

– To ensure that the ICT products placed on the market comply with all the requirements set out in the relevant legislation and regulations;

– To ensure that ICT products placed on the market do not cause electromagnetic interference, harm the public telecommunications network, and endanger health, safety, environment or any other aspect of protection of public interests;

– To take necessary action (e.g. prohibitions, withdrawals, recalls) to stop the circulation of products that do not comply with all the requirements set out in the relevant legislation and regulations, to bring the products into compliance and to apply sanctions.

Cameroon

One of the concerns expressed at the ITU World Assemblies is the compliance and interoperability (C&I) of ICT equipment placed on the market, particularly in developing countries.

The “Conformity and Interoperability” program, established by the Resolution 177 of PP-10, Resolution 76 of WTSA-12 and Resolution 47 of WTDC-10, aims to improve the ITU-T Recommendations by reducing the digital divide and That the standardization gap by providing them with the specialized tools and expertise they need to establish Regional C & I Test Centers.

It is based on four pillars:

1. Conformity assessment, which consists of assessing the degree of conformity of a product with ITU standards;

2. Interoperability events, based on the compatibility of the products of different vendors;

3. Capacity building, based on raising awareness among decision-makers and industry of the importance of compliance testing and interoperability;

4. Establishment of ICT testing centers in different regions of the world.

For effective and efficient implementation of these programs, priority should be given to capacity building based on raising awareness among decision-makers and contractors of the importance of compliance testing and interoperability.

ITU technical and financial assistance to developing countries is of great importance for the organization of seminars for this purpose.

Central African Republic

The regulatory authority of the Central African Republic[26] reported the importance of ensuring compliance and interoperability of ICT equipment. The challenge of enforcing the minimum provisions on equipment with regard to safety, health, quality, interoperability and compliance with frequency assignment in order to avoid interference in addition to combating counterfeit products. The major difficulty for the regulatory authority to fulfil its responsibilities is the lack of infrastructure to acquire testing laboratories. Another difficulty is the cost of training specialized personnel, who are able to operate in laboratories. Several technical officials from the regulatory authority were trained at the Center for Studies and Research in Telecommunications in Tunisia but failed to put into practice the skills acquired (because there are no laboratories in the RCA).

Strategies and proposals RCA:

– Identify the origin of the products or the legally responsible person;

– Identify the location of the factories;

– Identify internationally recognized certification authorities and laboratories;

– Create a sub-regional C&I test center.

Ghana

Conformity and interoperability have become a global issue and many administrations have or are now putting structures in place for that purpose. Under the laws of Ghana, the National Communications Authority is the sole Authority for ensuring that electronic communication equipment meet specific requirements before they are placed on the Ghanaian market. Over the past years Ghana has been going through a period of developing the process of assessing conformity of an electronic communication equipment in the absence of a testing laboratory. In case of conformity assessment procedures for Ghana, otherwise called the Type Approval Regime, a product must demonstrate that it meets a fourfold requirement per our various laws governing the Type Approval Regime.

The National Communications Authority (NCA) Act, 2008, Act 769 mandates the NCA to certify and ensure the testing of communications equipment for compliance with international standards, environmental health and safety standards including electromagnetic radiation and emissions. The Electronic Communications Act, 2008, Act 775 further iterates that such communications equipment to be certified must not pose any harm to the public communication network and must be compatible with the network.

In Ghana,[27] The National Communications Authority seeks to ensure that electronic communications equipment meets specific requirements before being placed on the market. The country has developed a conformity assessment process; however, it does not have a testing laboratory. Because of the need to control the domestic market for telecommunications products, NCA created a type approval scheme and incorporated temporary licensing, as well as involving border control authorities, importers and equipment manufacturers are dealing with the issue. It also held workshops to raise awareness among the public as well as the local media about the need to electronic communications equipment to comply with the standards.

Guinea

From the African continent, the use, link-up or connection and/or commercialization of any telecommunication equipment or materials within the Republic of Guinea[28] is subject to type approval or certification by ARTP. This procedure is in compliance with the provisions of Articles 8 and 13 of Law No. L/2005/018/AN of 8 September 2005, on the General Regulation of Telecommunications in the Republic of Guinea. In the Guinean contribution to Q4/2, it is mentioned that conformance evaluation contributes to protecting the quality of products in conformance with required standards:

– Safety of users;

– Safety of operating staff;

– Protection of the environment and the public.

In Guinea, although these provisions are regulated by a law, their application are still at a very early stage. The purpose of type approval is to check the conformance of an item of telecommunication equipment with the basic requirements applicable to it.

This conformance evaluation contributes to protecting:

– The quality of products in conformance with required standards:

• Safety of users;

• Safety of operating staff;

• Protection of the environment and the public.

In Guinea, although these provisions are regulated by a law, their application is still at a very early stage.

Indeed, the use, link-up or connection and/or commercialization of any telecommunication equipment or materials within the Republic of Guinea is subject to its type approval or certification by ARTP in compliance with the provisions of Articles 8 and 13 of Law No. L/2005/018/AN of 8 September 2005 on the General Regulation of Telecommunications in the Republic of Guinea.

The main drawbacks noted in the field may be summarized as follows:

– An active domestic market that escapes control owing to the permeability of borders;

– Technological neutrality and convergence which accelerate the development of new technologies and make the standardization process difficult;

– The constant refusal of players to comply with the type-approval rules.

The possible solutions now being envisaged, pending promulgation of the new law on telecommunications, are:

– Intervention by several players, including: ARPT, customs, taxation, ministries, etc.

– Training of national customs officers by ARPT on the visual or physical recognition of all telecommunication equipment or materials, and involvement of the consumer through awareness-raising campaigns;

– Implementation of the system for labelling type-approved equipment;

– Strict and synchronized monitoring of ITU’s standardization work;

– Surveillance of digital market through regular inspection missions;

– Support from ITU-T is essential in the form of targeted training;

– Harmonization of different regulations on labelling and type approval.

Haiti

Like several other less developed countries, Haiti[29] relies on the norms and conformance criteria made by developed country/ region to assess the conformance of the mobile terminals that must be evaluated. In Haiti homologation of mobile terminals matters because telecommunication in Haiti is mobile. In fact, since the 2010 earthquake wired structure are totally destroyed and less than 50,000 wired subscribers in a country of 6 million of mobile subscribers. Strengthening the actual regulation framework, capacity building and the establishment of Mutual Recognition Agreements are among the priorities in that country.

In Haiti,[30] the Conseil National des Telecommunications (CONATEL) was created by decree on September, 27, 1969. By another decree on June 10 of year 1987, CONATEL received full power to control the entrance in the country of all type of telecommunication equipment. In order to do so, CONATEL requires the type approval of all new telecom equipment and also inspects all imported telecom equipment.

Iran

Iran[31] presented a proposal for evaluation and type approval methods and procedures in the field of Information Technology (IT). It contains part of a study considering: the classification, required and suitable tests; evaluation and type approval methods; and procedures in the field of Information Technology (IT).

The study concludes that though many standards, procedures, laboratories, and regulatory institutions exist for testing or certifying the suitability of communication equipment, there is no common agreement or general understanding about the need for, and standards and procedures of IT equipment test, evaluation and type approval. This study intends to fill in this gap and help national regulatory bodies and test laboratories to evaluate IT equipment based on uniform and documented standards.

Mauritania

The scheme for the type approval of telecommunications equipment in Mauritania is organized by the decree No. R132 /MIPT related to standardization and certification of terminal equipment and pursuit of the activities of radio installations[32]. All terminal equipment intended to be connected, directly or indirectly, to a network open to the public may not be placed on the Mauritanian market only after approval. This approval is required prior to the placing on the market of any radio equipment, regardless of its destination.

The purpose of homologation is to verify the conformity of telecommunications equipment with the requirements applicable to it. This conformity assessment helps to protect: health; safety; environment and to ensure: the proper use of the radio spectrum; interoperability.

Challenges related to the nature of the activity:

– The emergence and rapid renewal of new technologies;

– A technological convergence which makes the standardization process more difficult;

– A dynamic internal market.

Possible solutions:

– Intervention of several actors in particular (ARE, Customs, Ministries, etc.);

– Consumer involvement through awareness campaigns;

– Market surveillance;

– The introduction of marking of approved equipment;

– ITU support is absolutely necessary in terms of capacity building and the establishment of a type approval laboratory.

Nepal

ICT standardization and conformity system in Nepal

Legislative provision for standardisation[33]

– Radio Act, 1957 and Radio Communication (License) Regulation, 1992.

– Telecommunication Act, 1997.

– Quality of Service (QoS) Benchmark of different Telecommunication Services (Fixed, Mobile and Internet).

– Type Approval Working Procedure for Radio Telecommunication Customer Premises Equipment, 2016 (TAP-04).

Type approval

– For radio telecommunication customer premises equipments (CPEs), NTA has been doing the type approval of radio telecommunication CPEs.

– All the concerned manufacturers/authorized agents/ representatives require applying and getting type approval for the radio telecommunication CPEs from NTA prior to import and/or sale in Nepal.

Equipment identified for type approval

– Customer site terminals that connect to public switched Network i. GSM/IMT-2000/ IMT Advanced. CDMA; GMPCS; and other telecom devices used to connect to mobile or PSTN network.

– Low Power Devices (LPDs) or Short Range Devices (SRDs). (WLAN – Wi-Fi, Bluetooth, and other 802.11x standard devices having 4W (max. EIRP) & 1 W Max. Transmitter output power, Frequency bands: 2.4 GHz, 5.1 GHz and 5.8 GHz).

Technical specification for mobile hand set

– Specific Absorption Rate: The handset shall have the SAR standard of maximum 2 W/Kg, averaged over 10 gm of tissue.

– Receiver Sensitivity: -102 dBm b. Maximum EIRP: 33+/-2 dBm.

– Minimum Charging Capacity: 600 mAh (Bar Phone and 1000 mAh(Smart Phone).

Equipment import autorisation

– The licensees of NTA need NTA’s recommendation to import infrastructure equipments for establishment and/or extension of network to provide telecommunication services.

– Such recommendations, when requested by the NTA’s licensees, are sent to the Ministry of Information and Communication who forwards its final recommendation to the Department of Customs or Department of Commerce as appropriate NTA has not yet adopted any standard operating procedure or has not formulated any guidelines based on which such process is carried out.

– During the process of such recommendation, technical specification will be evaluated to determine whether the operating frequency range is as per the spectrum assigned to the applicant (NTA’s licensee) or not and compliance of international standards e.g. ETSI, ITU or not.

– In case of microwave radio telecommunication equipment, frequency plan and network diagram will also be evaluated to determine whether the spot frequencies are approved by NTA or not.

Ongoing activities on ICT standardization and conformity system in Nepal

– Formulation of Regulatory framework of Standardization of Radio Telecommunication Equipment.

– Formulation of Regulatory framework of Type Approval/ Licensing of LPD/SRD.

– Formulation of Regulatory framework of E-Waste Management.

– Formulation of Regulatory framework of National Numbering Allocation, Assignment and Pricing.

– Formulation of directives/guidelines for implementation of national EIR.

– Formulation of bylaws of quality of service for telecommunication services.

Limitations of ICT standardization and conformity system in Nepal

– There is no laboratory inside the country to test the compliance of the requirement of type approval.

– The type approval of radio telecommunication CPEs is only based on document verification. NTA recognized certificates/test reports of international/national/ territorial standard bodies like a unilateral mutual recognized agreement (MRA).

– There is lack of human resources and technical expertise for ICT standardization and conformity system in Nepal.

– Only Customer site radio terminals that connect to public switched Network e.g. GSM, CDMA, IMT 2000, IMT Advanced, GMPCS etc. and LPDs/SRDs with operating frequency bands 2.4 GHz, 5.1 GHz and 5.8 GHz are facilitated for type approval.

– There are many LPDs/SRDs operating beyond frequency bands 2.4 GHz, 5.1 GHz and 5.8 GHz. There is no standard/benchmark like frequency range and RF output power level for such equipment which are essential for type approval.

Possible solutions

– ITU needs to facilitate to establish laboratory for conformance and interoperability.

– Capacity building training helps to enhance capacity and performance of human resources involved in conformance and interoperability and also bridges the standardization gap between developed country and developing country.

– ITU needs to encourage increasing participation from developing and under developed countries.

Laboratory CPqD

The contribution from Fundação CPqD (Brazil) presents an overview of the importance for society in carrying out laboratory tests aimed at conformity assessment of telecommunications products[34].

According to their experience, a local test laboratory contributes to the development of the national industry by providing inputs that enable project validation and improvement. In addition, a test laboratory promotes the growth of knowledge and supports the regulatory agencies in the certification process.

Among the main benefits, a laboratory:

– Enhances user safety and protection of consumer rights;

– Increases national industry competitiveness. Also it increases product quality and reliability as well as users satisfaction;

– Ensures that products commercialized or used in the country meet the minimum requirements (quality, safety, spectrum allocation, interoperability, etc.);

– Makes the entry of counterfeit products in the country more difficult by establishing tools that concretely enable enforcement activities;

– Provides human knowledge acquisition and technological transfer (e.g. on ICT technologies, testing methodologies, equipment configuration);

– Contributes to enhance human capacity and knowledge exchange with government agencies, universities and R&D centres.

4.3 Case Study Library

The ITU-D Study Groups shares the accumulated knowledge and experience of one participant with other members. These case studies were published as contributions to meetings in the following link: Case Study Library

Countries reported during ITU C&I events relevant information on C&I regulatory and policy aspects, challenges and approaches for tackling C&I issues:

Figure 11: Case studies reported in C&I events worldwide

Table 1: Case studies reported in C&I events worldwide

Countries

Argentina

El Salvador

Nepal

Bahamas

Gambia

Nicaragua

Bangladesh

Ghana

Papua New Guinea

Brazil

Guatemala

Paraguay

Brazil CPqD

Guinea

Peru

Burkina Faso

Haiti

Rwanda

Burundi

Honduras

Saint Lucia

Chad

India

Sierra Leone

Comoros

Jamaica

Singapore

Congo

Kenya

Sri Lanka

Costa Rica

Kingdom of Bahrain

Sudan

Cote d’Ivoire

Lesotho

Tanzania

Cuba

Malawi

Trinidad and Tobago

Dominican Republic

Mauritania

Uganda

ECTEL

Mexico

Venezuela

Ecuador

Mongolia

Zambia

C&I assessment studies on a regional basis

With the collaboration between Regional organizations for ICTs and ITU Regional Offices, the C&I assessment studies looks for promoting the establishment of Harmonized C&I Programmes, when possible. These activities are collaborating to improve regional integration and fostering the availability of highly qualified institutions, such as:

– Regional standardization organizations;

– Laboratories;

– Certification bodies;

– Accreditation bodies.

In an overall analysis, the Assessment Studies contributes to:

– Bridging the Standardization Gap;

– Reducing the Digital Divide; and

– As is inherent to ICT technologies development, strengthens business environment for global players.

As an example of a successful case, the Assessment Study on Conformance and Interoperability for Maghreb Countries has reached concrete deliverables, include the following recommendations, namely:

– Possible scenarios for establishing Common C&I Programme and Mutual Recognition Agreements (MRAs);

– Possible ways to facilitate the implementation of ITU Conformity and Interoperability programme;

– Programme especially in the areas of Capacity building and the Establishment of Test Centres on regional and sub-regional basis;

– Regional or sub-regional organizations suitable to perform conformity assessment testing including the function of MRA coordination;

– Entities that are capable of providing funding opportunities to support the establishment of testing centers in the region.

4.4 ITU-D Guidelines

A useful reference are the ITU Guidelines on Establishing Conformity and Interoperability Regimes, in which the following statements are important to note:

“The fundamental building block of an enforceable regime for addressing the establishment of an orderly telecommunication service and equipment marketplace is the enabling legislation. Many, if not most ITU Member States have established this legislation under various names and with varying levels of scope.

Such legislation may be titled in short form as Telecommunication Act, Radiocommunication Act, or created as a combined act including telecommunications, radio communications, and possibly other elements such as metrology and principles for establishment of fees for services aspects. Such Acts are “the law of the land” and are further interpreted by regulatory requirements that deal with such practical matters as penalties for infractions, setting fees, obligations of parties, importation, market surveillance and so on”.

“A Telecommunication Act reflects the policy of the sovereign state in question and can include a clear statement of the underlying policy. This statement would cover such elements as the following:

– Orderly development of a telecommunications system;

– Reliable and affordable telecommunications services of high quality;

– Highlight the role of telecommunications to enhance efficiency and competitiveness;

– Ensure that regulation, where required, is efficient and effective;

– Stimulate research and development and encourage innovation in the provision of telecommunications services;

– Respond to the economic and social requirements of users of telecommunications services;

– Contribute to the protection of the privacy of persons.

List of guidelines published:

Guidelines for developing countries on Establishing Conformity assessment Test Labs in Different Regions (2012)

This set of guidelines is the first publication on C&I, its valuable content includes information concerning: The process required for building testing labs; a site analysis (e.g., existing testing labs, know-how); collaboration mechanisms; best practices; reference standards and ITU Recommendations.

Guidelines for the Development, Implementation and Management of Mutual Recognition Arrangements/Agreements on Conformity Assessment (2013)

These guidelines promote the understanding and establishment of Mutual Recognition Agreements (MRAs) on conformity assessment that are intended to promote efficiency and resource sharing as well as to streamline the flow of products among participating Parties such as ITU Member States and private sector organizations, such as testing laboratories

Feasibility Study for the establishment of a Conformance Testing Centre (2013)

This feasibility study describes environments, procedures and methodologies to be adopted to establish, manage and maintain a testing center covering different kinds of conformance and interoperability testing areas

Establishing Conformity and Interoperability Regimes – Basic Guidelines (2014)

These Guidelines address challenges faced by developing countries as they plan and review their own C&I regimes. Aspects covered by this publication include, inter alia, conformity assessment procedures; legislation to promote an orderly equipment marketplace; surveillance; coordination across regulatory agencies; and relevant international standards.

Establishing Conformity and Interoperability Regimes – Complete Guidelines (2015)

These Guidelines compiled from a careful collection of international best practices, address challenges faced by developing countries as they plan and review their own C&I regimes. Aspects covered by this publication include, inter alia, conformity assessment procedures; the right type of approval system, the legislation required to promote an orderly telecommunication service and marketplace; the calculation of fees, and the ideal enforcement and Surveillance; coordination across regulatory agencies; relevant international standards.

4.5 Recommendations for assessment studies on a regional basis

Objective of the C&I assessment study

The objective is to undertake C&I assessments on a regional basis. The project aims to identify all the necessary elements and promote the collaboration among regional and S=sub-regional organizations for establishing a common C&I regime and mutual recognition agreements, seeking to present possible scenarios to meet the needs and interests of Member States and the regions.

Organizations to conduct the C&I assessment study must consider trusted regional associations or advisory bodies in telecoms or ICTs. The assessment can cover but are not be limited to:

General aspects of the region:

– Description of the region e.g., demography, economy, geography, penetration of telecoms and Internet including wireless, broadband and ICTs, governance, service providers, supply and manufacturing, natural resources and export/imports;

– Countries involved in the study;

– Identification of LDC or LIC members in the region.

Regulatory framework and institutions

– Regulatory framework and regulation which establishes technical requirements for products and services to be legally imported and deployed in the marketplace e.g. ICT products and services, electrical apparatus, environmental requirements, etc.);

– Conformity assessment schemes adopted for market entry (certification, self-declaration, labeling, use of proxies such as EC, FCC or others etc.). Knowledge of ISO/CASCO set of Guidelines and standards;

– Legislation and regulation dealing with ICT and telecom products and services and related areas such as electrical safety and environmental issues, how is it applied? Is it compulsory or voluntary;

– Delegation of authorities to foreign entities under arrangements such as MRAs on Conformity Assessment e.g. for certification;

– National standards system and national SDOs;

– Metrology legislation and any National Institute of metrology responsible to maintain the national measurement standards in the country; to establish and maintain their metrological traceability to the official system of units;

– Possible resources from national/regional/international funds to assist private and public sector to investment in infrastructure, e.g., labs and human resources;

– Importation control of the products entering the country/region enforced e.g. at point of entry, spot checks and post market surveillance;

– Post market surveillance, audit and enforcement regime established for products entering the country/region, and deployed in the country/region, and a schedule of punishments for infractions;

– Actions, if any, undertaken to identify counterfeit products and what actions are taken to remove such products from the marketplace and to deal with parties responsible for bringing them into, or deploying them in the country/region.

Accreditation

– Accreditation body (ISO/IEC 17011) (not only in ICT) established;

– Accreditation scopes.

Laboratories

– Laboratories identified in the country/region and what service levels do they provide (e.g. 1st, 2nd and 3rd party testing);

– Laboratories accredited (ISO 17025) or if there is any kind of peer evaluation of the lab;

– Testing scopes of such Labs.

Certification bodies and marking

– Certification bodies (ISO/IEC 17065) in the country;

– Scopes of the certification bodies in ICTs and telecom;

– Marks of conformity for ICT products in the country/region that are trusted.

Supplier’s declarations of conformity

– Declarations in accordance with ISO/IEC 17050;

– May require a technical file to support declaration be made available to authorities;

– May require declaration information in accompanying documents.

Establishing a common C&I regime and MRAs

– The final work shall specify possible Scenarios to permit the collaboration for establishing a common C&I regime and mutual recognition agreements;

– Identification of the organizations in the country/region that can lead the de establishment and maintenance of a MRA or other technical collaboration.

5 CHAPTER 5 – Roadmap for Member States for implementing C&I programs

While there is not a one-size-fits-all solution for solving C&I issues, the chart below organizes a roadmap for developing countries considering to adopt actions to improve the level of conformity and interoperability. These general guidelines are identified in the previous chapters of these Report and are recalled as follows:

Recommendations – Summary of the chapters

Capacity building

Enhancing awareness on C&I issues

C&I Procedures:

– Type Approval;

– Standards;

– Regulatory issues;

– Others.

Fields of study :

– C&I Testing Domains : EMC, mobile, NGN, batteries, DTV receivers, etc.;

– Laboratory accreditation;

– Calibration;

– Virtual testing;

– C&I regimes:

• Policy aspects;

• Regulation: rules and regulation;

• Conformity Assessment Schemes;

• Certification procedures;

• MRA;

• Post-Market Surveillance.

Knowledge share, collaboration, and Mutual Recognition Agreements

Platform for cooperation among the experts from the international community in charge of C&I

Legislation and regulation

Drafting of regulation

Public consultation

Definition of Implementation phases

Follow-up of the benefits achieved and lessons learned

Revision and improvements

Technical requirements and standards

Drafting new technical requirements

Adoption

Harmonization at Regional and International levels

Conformity Assessment (CA) schemes

Establishment of Conformity Assessment Institutions

Collaboration with International CA Institutions

Laboratory services

Establishment of laboratories

Funding

Cost estimation at local level

Defining testing domain priority according to local urgent needs

Sharing of testing facilities

Use of regional testing centers

Virtual laboratories

Customized assistance

Recommended for developing countries in need of specific assistance in C&I

[1] WSIS 2016 Outcomes: http://www.itu.int/net4/wsis/forum/2016/Outcomes/#hlt; WSIS Action Lines Supporting Implementation of the SDGs: http://www.itu.int/net4/ wsis/ forum/2016/ Content/documents/outcomes/ WSISForum2016%E2%80%94 WSISActionLines SupportingImplementationSDGs.pdf.

[2] Between products in a multi-vendor, multi-network and multi-service environment.

[3] Such as ETSI, 3GPP, ITU, etc.

[4] From the WSIS 2016 Outcomes: “ICTs have clearly demonstrated their value as cross-cutting facilitators and enablers of sustainable development.” Also ISO standards help meet SDGs says World Bank Group expert: http://www.iso.org/iso/home/news_index/news_archive/news.htm?refid=Ref2118.

[5] The full listing of standards are available at: http://www.iso.org/iso/home/store/catalogue_tc/catalogue_tc_browse.htm?commid=54998&includesc=true&published=on&development=on.

[6] Additional information on CASCO is available at http://www.iso.org/iso/home/about/conformity-assessment/casco.htm.

[7] SIP: Session Initiation Protocol, a common standardized VoIP protocol for VoIP.

[8] Document 2/224, “Contribution of laboratories to C&I”, Fundação CPqD – Centro de Pesquisa e Desenvolvimento em Telecomunicações, Federative Republic of Brazil.

[9] Document SG2RGQ/133, “BDT activities conducted for the implementation of the ITU C&I Programme”, BDT Focal Point for Question 4/2.

[10] APEC – Asia-Pacific Economic Cooperation: http://www.apec.org/groups/som-steering-committee-on-economic-and-technical-cooperation/Working-groups/telecommunications-and-Information/apec_tel-mra.aspx.

[11] CITEL – Inter-American Telecommunication Commission: http://www.citel.oas.org.

[12] Document SG2RGQ/148, “Rio 2016 Games – ICT equipment market surveillance: practices, figures and facts”, Federative Republic of Brazil.

[13] Document SG2RGQ/248, “Simplified testing procedures - a study case used during the major events in Brazil”, Fundação CPqD – Centro de Pesquisa e Desenvolvimento em Telecomunicações, Federative Republic of Brazil.

[14] Document SG2RGQ/161, “Conformance testing - affordable solutions - virtual laboratories”, Fundação CPqD – Centro de Pesquisa e Desenvolvimento em Telecomunicações, Federative Republic of Brazil.

[15] Document 2/353, “C&I Activities in Central America”, COMTELCA.

[16] 2nd Workshop on Conformity and Interoperability for COMTELCA member countries: Borrador en abierto: http://www.itu.int/en/ITU-D/Regional-Presence/Americas/Documents/EVENTS/2016/15556-NI/Borrador%20Abierto-v3-7-December2016.pdf.

[17] E.g. Europe http://ec.europa.eu/growth/single-market/goods/building-blocks/market-surveillance/organisation/, Canada http://www.ic.gc.ca/eic/site/mra-arm.nsf/eng/h_nj00055.html#market, and Japan http://incompliancemag.com/article/electronic-product-compliance-in-japan/.

[18] Document SG2RGQ/149, “Post market surveillance: practices, figures and fact”, Federative Republic of Brazil.

[19] Document SG1RGQ/225, “Post Market Surveillance - an example of procedures from a real case”, Federative Republic of Brazil.

[20] Document 2/224 + Annex, “Contribution of laboratories to C&I”, Fundação CPqD – Centro de Pesquisa e Desenvolvimento em Telecomunicações, Federative Republic of Brazil.

[21] Document SG2RGQ/75, “Follow-up on activities from conformity and interoperability - the Caribbean Workshop“, Caribbean Telecommunications Union.

[22] Document 2/52, “Brazilian Case Study on C&I”, Federative Republic of Brazil and Document SG2RGQ/48, “The certification of telecommunications products in Brazil”, Federative Republic of Brazil.

[23] Document 2/52, “Brazilian Case Study on C&I”, Federative Republic of Brazil and Document SG2RGQ/48, “The certification of telecommunications products in Brazil”, Federative Republic of Brazil.

[24] Document 2/79, “Conformité et interopérabilité des systèmes”, Republic of Cameroon.

[25] Document 2/236 + Annex, “Market surveillance – global debates, best practices and examples from the field”, Federative Republic of Brazil.

[26] Document 2/304, “Assistance to developing countries for implementing conformance and interoperability programmes”, Central African Republic.

[27] Document 2/39, “Conformity assessment procedures in Ghana”, Ghana.

[28] Document 2/166, “Assistance to developing countries for implementing conformance and interoperability programmes”, Republic of Guinea.

[29] Document SG2RGQ/139, “Conformance evaluation and interoperability of mobiles terminals in less developed countries: the Haitian case”, Republic of Haiti.

[30] Document 2/227 + Annex, “Haiti homologation process and challenges”, Republic of Haiti.

[31] Document 2/343 + Annex, “Type approval and test of Information Technology Equipment”, Islamic Republic of Iran.

[32] Document SG2RGQ/61, “Homologation of telecommunication equipment: the experience of Mauritania”, Islamic Republic of Mauritania.

[33] Document SG2RGQ/240, “Status of conformity and interoperability in Nepal”, Nepal Telecommunications Authority (NTA), Republic of Nepal.

[34] Document 2/224 + Annex, “Contribution of laboratories to C&I”, Fundação CPqD – Centro de Pesquisa e Desenvolvimento em Telecomunicações, Federative Republic of Brazil.

Abbreviations and acronyms

Various abbreviations and acronyms are used throughout the Report, they are provided here.

Abbreviation/acronyms

Description

AB

Accreditation Body

APEC

Asia-Pacific Economic Cooperation

ATM

Abstract test method

ATS

Abstract test suite

BDT

Telecommunication Development Bureau of ITU

C&I

Conformance and Interoperability

C&I

Conformance and Interoperability

CAB

Conformity Assessment Body

CASCO

ISO committee on conformity assessment

CB

Certification Body

CITEL

Inter-American Telecommunication Commission

CPqD

Centro de Pesquisa e Desenvolvimento em Telecomunicacoes

GLP

Good Laboratory Practice

IAAC

InterAmerican Accreditation Cooperation

IAF

International Accreditation Forum

ICT

Information and Communications Technologies

IEC

International Electrotechnical Commission

IECEE CB

IEC System for conformity testing and certification of electrical and electronic components, equipment and products certification body

ILAC

International Laboratory Accreditation Cooperation

ISO

International Standardization Organization

IT

Information Technology

ITU

International Telecommunication Union

ITU-D

ITU Telecommunication Development Sector

IUT

Implementation Under Test

LDC

Least Developed Countries

MRA

Mutual Recognition Agreement

NCA

National Communications Authority

PICS

Protocol Implementation Conformance Statement

QoS

Quality of Service

RCB

Regional Certification Body

SAR

Specific Absorption Rate

SDO

Standards Development Organization

SDoC

Supplier Declaration of Conformity

SIDS

Small Island Developing States

SIP

Session Initiation Protocol

TBT

Agreement on Technical Barriers to Trade

TTCN

Tree and Tabular Combined Notation

WSIS

World Summit on the Information Society

WTDC

World Telecommunication Development Conference

C&I Vocabulary

The following definitions are used in the context of this Report:

C&I Vocabulary

Description

Abstract test method (ATM)

The description of how an Implementation Under Test (IUT) is to be tested, given at an appropriate level of abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable abstract test cases to be specified for this test method [ITU-T X.290]

Abstract test case

A complete and independent specification of the actions required to achieve a specific test purpose, defined at the level of abstraction of a particular Abstract Test Method, starting in a stable testing state and ending in a stable testing state. This specification may involve one or more consecutive or concurrent connections [ITU-T X.290]

Abstract test suite (ATS)

A test suite composed of abstract test cases [ITU-T X.290]

Acceptance or acceptance of conformity assessment results

Use of a conformity assessment result provided by another person or body. [ISO 17000]

Accreditation

Third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks. [ISO 17000]

Accreditation body

Authoritative body that performs accreditation. [ISO 17000]

Approval

Permission for a product or process to be marketed or used for stated purposes or under stated conditions. [ISO 17000]

Attestation

Issue of a statement, based on a decision following review, that fulfilment of specified requirements has been demonstrated. [ISO 17000]

Basic interconnection test

A test of an IUT which has limited scope to determine whether or not there is sufficient conformance to the relevant protocol(s) for interconnection to be possible, without trying to perform thorough testing. [ITU-T X.290]

Bilateral arrangement

Arrangement whereby two parties recognize or accept each other’s conformity assessment results. [ISO 17000]

Certification

Third-party attestation related to products, processes, systems or persons. [ISO 17000]

Conformance

Compliance with requirements specified in applicable series Recommendations. [ITU-T X.290]

Conformity assessment

Demonstration that specified requirements relating to a product, process, system, person or body are fulfilled. [ISO 17000]

Conformity assessment body

Body that performs conformity assessment services. [ISO 17000]

Conformity assessment scheme (or programme)

Conformity assessment system related to specified objects of conformity assessment, to which the same specified requirements, specific rules and procedures apply. [ISO 17000]

Declaration

First-party attestation. [ISO 17000]

Designating body

Means a body appointed by a Party, with responsibility to identify and monitor testing laboratories and/or certification bodies. [APECTEL]

Electromagnetic compatibility

The ability of an equipment or system to function satisfactorily in its electromagnetic environment without introducing intolerable electromagnetic disturbance to anything in that environment. [IEC 60050 and ITU-T K.63]

Equal treatment

Treatment accorded to products or processes from one supplier that is no less favourable than that accorded to like products or processes from any other supplier, in a comparable situation. [ISO 17000]

Equal and national treatment

Treatment accorded to products or processes originating in other countries that is no less favourable than that accorded to like products or processes of national origin, or originating in any other country, in a comparable situation. [ISO 17000]

First-party conformity assessment activity

Conformity assessment activity that is performed by the person or organization that provides the object. [ISO 17000]

Implementation under test

An implementation of one or more OSI protocols in an adjacent user/provider relationship, being that part of a real open system which is to be studied by testing. [ITU-T X.290]

Implementation conformance statement

A statement made by the supplier of an implementation or system claimed to conform to a given specification, stating which capabilities have been implemented. The ICS can take several forms: protocol ICS, profile ICS, and information object ICS. [ITU-T X.290]

Inspection

Examination of a product design, product, process or installation and determination of its conformity with specific requirements or, on the basis of professional judgement, with general requirements. [ISO 17000]

Interoperability

The ability of two or more systems or applications to exchange information and to mutually use the information that has been exchanged. [ITU-T Y.101]

Interoperability (Management Network)

The ability of network management products and services from different suppliers to work together to manage communications between managed object classes. [ITU-T M.80]

Interoperability testing

Testing to assess the ability of two or more systems to exchange information and to make mutual use of the information that has been exchanged. [ITU-T Z.450]

Homologation

Recognition by the national authority that certain ICT equipment complies with the technical regulation in place

Market surveillance

Activities carried out and measures taken by public authorities to ensure that products comply with the requirements set out in the relevant Community harmonisation legislation and do not endanger health, safety or any other aspect of public interest protection. [EU 765/2008/EC]

Model network

network which simulates the capabilities similar to those available in present telecommunication networks, has a similar architecture and functionality and uses the same telecommunication technical means [ITU-T Q.3900]

Most favoured nation

Countries cannot normally discriminate between their trading partners, where one is granted a special favour (such as a lower customs duty rate for one of their products) than all other members must receive the same favour. [WTO]

Multilateral arrangement

Arrangement whereby more than two parties recognize or accept one another’s conformity assessment results. [ISO 17000]

Mutual recognition agreement

A formal legal commitment between parties for recognition of conformity assessment results for telecommunication equipment. [ITU Guidelines on MRA]

Mutual recognition arrangement

A voluntary arrangement (procedures and processes) between parties for recognition of conformity assessment results for telecommunication equipment [ITU Guidelines on MRA]

Mutual recognition agreement – Phase 1

Mutual recognition of testing laboratories and mutual acceptance of test reports prepared by the testing laboratories

Mutual recognition agreement – Phase 2

Mutual recognition of certification bodies and mutual acceptance of certification prepared by the certification bodies

National treatment

Treatment accorded to products or processes originating in other countries that is no less favourable than that accorded to like products or processes of national origin, in a comparable situation. [ISO 17000]

National accreditation body

The sole body in a Member State that performs accreditation with authority derived from the State. [EU 765/2008/EC]

Next generation network (NGN)

A packet-based network able to provide Telecommunication Services to users and able to make use of multiple broadband, QoS-enabled transport technologies and in which service-related functions are independent of the underlying transport-related technologies. It enables unfettered access for users to networks and to competing service providers and services of their choice. It supports generalised mobility which will allow consistent and ubiquitous provision of services to users. [ITU-T Recommendation Y.2001]

NGN technical means

The NGN basic equipment which serves as a basis for building new generation network solutions, including for application in public telecommunication networks [ITU-T Q.3900]

NGN monitoring systems (NMS)

A system which is responsible for online (under payload) measurement values of the NGN protocols carried out on the different NGN strata [ITU-T Q.3902]

Peer evaluation

A process for the assessment of a national accreditation body by other national accreditation bodies. [EU 765/2008/EC]

Protocol implementation conformance statement (PICS)

An Implmentation Conformance Statement (ICS) for an implementation or system claimed to conform to a given protocol specification [ITU-T X.296]

Pluri-lateral agreement

An agreement which only some members have signed. [WTO]

Product certification

An activity by which a third party gives written assurance that a product (including process and service) fulfils specified requirements. [ISO Guide 67]

Review

Verification of the suitability, adequacy and effectiveness of selection and determination activities, and the results of these activities, with regard to fulfilment of specified requirements by an object of conformity assessment. [ISO 17000]

Recognition or recognition of conformity assessment results

Acknowledgement of the validity of a conformity assessment result provided by another person or body. [ISO 17000]

Scope of attestation

Range or characteristics of objects of conformity assessment covered by attestation. [ISO 17000]

Second-party conformity assessment

Activity conformity assessment activity that is performed by a person or organization that has a user interest in the object. [ISO 17000]

Specified requirement

Need or expectation that is stated. [ISO 17000]

Standard

Document approved by a recognized body, that provides, for common and repeated use, rules, guidelines or characteristics for products or related processes and production methods, with which compliance is not mandatory.  It may also include or deal exclusively with terminology, symbols, packaging, marking or labelling requirements as they apply to a product, process or production method. [WTO TBT Agreement]

Supplier’s declaration of conformity

Is a “declaration” as defined in ISO/IEC 17000, i.e. first-party attestation. [ISO 17050] (Note. To avoid any confusion with attestation by certification bodies, the term “self-certification” is deprecated and should not be used.)

Surveillance

Systematic iteration of conformity assessment activities as a basis for maintaining the validity of the statement of conformity. [ISO 17000]

System under test (SUT)

the real open system in which the IUT resides [ITU-T X.290]

Technical regulation

Document which lays down product characteristics or their related processes and production methods, including the applicable administrative provisions, with which compliance is mandatory.  It may also include or deal exclusively with terminology, symbols, packaging, marking or labelling requirements as they apply to a product, process or production method. [WTO TBT Agreement]

Technical requirements

Set of product characteristics defined by a technical regulation.

Test laboratory

An organization that carries out conformance testing. This can be a third party, a user organization, a telecommunications administration or recognized private operating agency, or an identifiable part of a supplier organization. [ITU-T X.290]

Test purpose (TP)

A prose description of a well defined objective of testing, focusing on a single conformance requirement or a set of related conformance requirements as specified in the appropriate OSI specification (e.g. verifying the support of a specific value of a specific parameter) [ITU-T X.290]

Test suite

A complete set of test cases, possibly combined into nested test groups, that is needed to perform dynamic conformance testing for one or more OSI protocols. [ITU-T X.290]

Testing

Determination of one or more characteristics of an object of conformity assessment, according to a procedure. [ISO 17000]

Third-party conformity assessment activity

Conformity assessment activity that is performed by a person or body that is independent of the person or organization that provides the object, and of user interests in that object. [ISO 17000]

Type approval

ee approval

Unilateral arrangement

Arrangement whereby one party recognizes or accepts the conformity assessment results of another party. [ISO 17000]

Quality of Service (QoS)

Totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service [ITU-T E.800]

QoS experienced/perceived by customer/user (QoE)

A statement expressing the level of quality that customers/users believe they have experienced [ITU-T E.800]

Annexes

Annex 1: Conformity and Interoperability practices

1.1 Identifying C&I capacity building needs – Questionnaire to assess and plan C&I trainings

The completion of this Section is optional

Name of Participant:

Company/Institution:

E-mail:

Your appreciation (decreasing from 6-excellent to 1-poor)

Issues

6

5

4

3

2

1

ITU administrative procedures prior to the training

⃝

⃝

⃝

⃝

⃝

⃝

Technical level of the training

⃝

⃝

⃝

⃝

⃝

⃝

Professional level and academic ability of Instructor

⃝

⃝

⃝

⃝

⃝

⃝

Methodology used for knowledge transfer

⃝

⃝

⃝

⃝

⃝

⃝

Quality of instructional material provided

⃝

⃝

⃝

⃝

⃝

⃝

Level of interaction: a) among participants

⃝

⃝

⃝

⃝

⃝

⃝

b) participants/Instructor

⃝

⃝

⃝

⃝

⃝

⃝

Achievement of goals established for the training

⃝

⃝

⃝

⃝

⃝

⃝

Activities carried out in the Labs (hands-on practices)

⃝

⃝

⃝

⃝

⃝

⃝

Logistics provided by Institution (lunch, transp., coffee breaks)

⃝

⃝

⃝

⃝

⃝

⃝

Accessibility/mobility within the Laboratory
premises

⃝

⃝

⃝

⃝

⃝

⃝

Infrastructure of Laboratory Partner

⃝

⃝

⃝

⃝

⃝

⃝

Positive and Negative Comments

+

-

Next Conformance and Interoperability training of your interest

Priority

Broad-
band

Elect-
rical
Protec-
tion

Inter-
opera-
bility

Mobile
and
wireless
network

Next
Gene-
ration
Network-
NGN

Optical
net-
works

Safety

Virtual
lab

1st

⃝

⃝

⃝

⃝

⃝

⃝

⃝

⃝

2nd

⃝

⃝

⃝

⃝

⃝

⃝

⃝

⃝

3rd

⃝

⃝

⃝

⃝

⃝

⃝

⃝

⃝

4th

⃝

⃝

⃝

⃝

⃝

⃝

⃝

⃝

For ITU Guidance, please indicate other training needs of your country/institution

1.2 Test by sampling and labelling

The following procedures are based on contribution SG2RGQ/248, from Brazil:

1.0. Testing and Labeling Methodology (T & E)

1.0.1. In order to perform the tagging of an equipment, the Supervisory Agent must perform the following activities:

I – Request the operating license or, if the equipment is homologated for restricted radiation, and consult the Certification and Homologation Management System;

II – Perform a visual inspection of the equipment presented, to check for obvious flaws or visible changes;

III – Verify the licensed stations for the use of the spectrum in that region;

IV – Perform technical measurements;

V – Take note of the serial numbers and any existing labels; and

VI – Update the T&E database.

1.1. Testing by sampling

1.1.1. In cases where the same entity has a large number of similar devices, only a few units, according to the sampling described in the following table:

Lot Size

Sample Size

Action Failed

<= 20

Test all

n / a

21 - 90

20

Test other 20

91 - 150

32

Test other 32

151 - 280

50

Test other 50

281 - 500

80

Test other 80

501 - 1200

125

Test other 125

1201 - 3200

200

Test other 200

1.1.2. As an example, suppose a company has 132 similar radios. In this scenario, 32 radios will be tested and if they do not problems during the test, the other 100 will not need to be tested, but if at least one of the radios does not pass the test, a new sample with additional 32 units will be tested until any sample of the equipment presents problems in the test or until all radios have been tested.

1.1.3. The equipment used for conformity testing procedures shall preferably be as follows:

I – FSVR spectrum analyzer;

II – FSL6 spectrum analyzer;

III – Telescopic antennas;

IV – Horn Antenna;

V – Cable box and adapters;

VI – Frequencimeter;

VII – Photographic camera; and

VIII – Coaxial load of 50Ω.

1.1.4. The spectrum analyzers mentioned above and the frequency counter must be connected and connected to 50Ω coaxial loads.

1.1.5. The “Reference Level” value must be adjusted so that the peak of the evaluated carrier does not exceed the limit of the screen, nor the 1dB compression “of the mixer so as not to generate” FI “overhead in the analyzer”.

1.1.6. The values for “VBW”, “RBW” and attenuation shall remain with the automatic configuration, unless other values of these settings are more efficient for measuring the parameters of certain equipment.

1.1.7. The authorization submitted by the entity must be consulted and the frequency and bandwidth licensed.

1.1.8. The FSL6 spectrum analyzer shall be set with the center frequency equal to that authorized for the entity and the SPAN configuration for the double the amount of authorized bandwidth.

1.1.9. The second FSVR spectrum analyzer must be set to the center frequency equal to that authorized for the entity and configure the SPAN for a value slightly higher than the double of the value of the frequency that is being tested in order to verify the occurrence of spurious or harmonics in a band without the need to change the configuration of the first spectrum analyzer.

1.1.10. The push-button (PTT) of the device must be pressed or the equipment must be connected, as appropriate, and checked on the analyzer if the frequency of the carrier is within what has been authorized.

1.1.11. The measured frequency value must be checked on the frequency counter.

1.1.12. Then, the FSL6 spectrum analyzer must be set to max hold mode and repeat the procedure described in item 1.2.10 for checking the bandwidth used by the equipment, compare it to the authorized value and record the measured value. At the same time, the SPAN of the FSVR spectrum analyzer should be gradually reduced until it is the same as that of the FLS6, while the existence of spurious signals is observed in more detail.

1.1.13. If there is more than one authorized frequency for the entity, repeat the procedures in items 1.2.7 to 1.2.12 until all have been tested.

1.1.14. For each equipment tested and complying with the authorized parameters, the corresponding label must be affixed, respecting the color mapping for each event location.

1.1.15. In cases where the equipment has not complied with the authorized parameters, the representative of the entity shall be allowed to reconfigure the equipment immediately.

1.1.16. In all cases where an equipment does not meet the compliance test, for this particular unit, it must be repeated.

1.1.17. The “not allowed use” label will be affixed to the tested equipment which:

I – Even after the procedures described in items 1.2.14 and 1.2.15, it did not operate according to the authorized parameters;

II – Present evidence of external technical changes;

III – Be forbidden (jammers, for example);

IV – Show emission of spurious or harmonic with intensity above the allowed;

V – Being of restricted radiation, does not have the Regulatory Authority homologation or authorization of temporary use.

1.1.18. At times when there is queuing and it is convenient to expedite the process, in order to avoid damages to the event, it may be performed a simplified test procedure, measuring only the frequency of operation of the equipment.

1.1.19. For the labeling of equipment approved for restricted radiation, it shall be observed if its operating frequency does not coincide with the licensed to operate on site. In addition, it should be checked in the T & E Table if other restricted radiation equipment, already tested for the event, is running at the same frequency. In this case, you should be asked to change the configuration of the equipment, so that it operates at a free frequency.

Annex 2: Relevant Recommendations and Reports of the other ITU sectors

1.1 Overview of ITU’s work to conformity and interoperability

Conformity with international standards, such as ITU Recommendations, is one of the core principles underlying the global interoperability of ICT networks, devices and services.

The ITU Conformity and Interoperability (C&I) programme was initiated at the request of ITU’s membership to enhance the conformity and interoperability of ICT products implementing ITU Recommendations or part thereof, solicit feedback to improve the quality of ITU Recommendations, and reduce the digital divide and the Standardization Gap, by assisting developing countries with human resource and infrastructure capacity building.

The ITU C&I Programme is organized in accordance with the ITU Plenipotentiary Conference Resolution 177 in four pillars (since Guadalajara, 2010), with ITU-T taking lead responsibility for Pillars 1 and 2, and ITU-D for Pillars 3 and 4. These four pillars are: 1) conformity assessment, 2) interoperability events, 3) human resource capacity building, and 4) assistance in the establishment of test centres and C&I programmes in developing countries.

While ITU-R is not prominent in the ITU C&I programme, it does create ITU-R Recommendations and Reports that guide testing of conformity to the specifications documented in other ITU-R Recommendations.

The remainder of this annex describes the ITU-T activities related to conformity assessment and interoperability events, then lists the ITU-R and ITU-T documents related to conformity and interoperability.

1.2 ITU-T Activities related to conformity assessment

ITU-T Study Group 11 (SG11) was designated by WTSA-12 as a lead ITU-T Study Group on test specifications, conformance and interoperability testing. The role of SG11 in this domains was strengthened by WTSA-16. SG11 coordinates ITU-T C&I activities across all ITU-T SGs.

SG11 has achieved the following important decisions:

– Approved the SG11 C&I action plan, based on the ITU C&I action plan approved by Council-12 and revised by Council 13;

– Established the Conformity Assessment Steering Committee (ITU-T CASC) to elaborate detailed procedures for the implementation of a test laboratory recognition procedure in ITU-T, documented in the ITU-T SG11 Guideline, “Testing laboratories recognition procedure”;

– Developed a living list of Recommendations and related testing specifications within key technologies suitable for conformance and interoperability testing and requested all study groups to submit a living list of technologies under study which are suitable for testing;

– Established collaboration with ETSI TC INT to develop standards in SIP-IMS conformity testing, Internet speed measurement, framework of an interconnection among VoLTE/ViLTE-based networks, requirements and test specifications for signalling protocols to be used for VoLTE/ViLTE interconnection;

– Started a new work item Q.30xx_VoLTE_Interconnection “Framework of interconnection of VoLTE/ViLTE-based networks” following the discussion at the ITU Workshop on “Voice and Video Services Interoperability Over Fixed-Mobile Hybrid Environments, Including IMT-Advanced (LTE)” on 1 December 2015. The development of test specifications will follow;

– Agreed on a standardization work plan for SIP-IMS conformance testing. It includes requirements and relevant test specifications for basic call and some supplementary services, which are used on IMS-based networks;

– Consented a new Recommendation ITU-T Q.3960 “Framework of Internet speed measurements for the fixed and mobile networks” which is the first of a series of ITU-T Recommendations on Internet speed measurement (link). This framework specifies guiding principles to establish a standardized architecture for national regulators to assess speed of Internet connection at the national and international levels;

– Initiated collaboration between SG11 and OECD aiming to explore the future adoption of an ITU framework that can be used for regulation of the broadband speed access connection. The detailed information about this activity is available at http://www.itu.int/en/ITU-T/C-I/Pages/IM/Internet-speed.aspx;

– Started a new pilot project “Mobile network portability (ITU-T Q.Suppl.4)” related to the C&I Programme in collaboration with SG2. The list of ongoing pilot projects is available http://www.itu.int/en/ITU-T/C-I/Pages/CI-projects-table.aspx;

– Agreed upon a work plan on benchmarking of IMS platform;

– requested all ITU-T SGs to develop test requirements for their current/future Recommendations as appropriate, and to update the list of ITU-T Recommendations to be tested for conformance and interoperability, including those that other standards organizations and forums have prepared.

Other ITU-T SGs have also been engaged in conformity assessment activities, mostly related to developing testing specifications:

– SG2 started developing test specifications for Rec. ITU-T M.3170 and started related pilot project;

– SG16 has developed many specifications to assist developers in checking compliance to ITU-T Recommendations, in particular for IPTV systems, voice compression and video compression, andcontinues updating ITU Recommendations related to interoperability compliance testing of personal health systems;

– SG5 developed resistibility tests[35] for telecommunication equipment and test specifications related to universal charge adapter[36] among other recommendations related to electromagnetic disturbance;

– SG12 developed test specifications for the universal wired headset, , and has revised Recommendation ITU-T P.1100/P.1110, based on the testing results of the first ITU-T test event on performance assessment of mobile phones in conjunction with hands-free telephone systems in a car. SG12 is also working on the conformance test specifications for voice over IP transmission quality;[37]

– SG15 is working on conformance and interoperability test plans for the optical network unit management control interface for Ethernet-based, plastic optical networks;

– SG17 maintain the Recommendations in the ITU-T Z.16x series defining testing and control notation.

The ITU Product Conformity Database was launched on 18 December 2014 to publicize the conformance of ICT products and services with ITU-T Recommendations.

ITU, IEC and ISO jointly organized a World Standardization Coordination Workshop on Conformity Assessment on 1-2 December 2015 in conjunction with the UNECE WP 6 meeting to promote and increase the worldwide visibility of international consensus-based standardization and related conformity assessment matters.

For promoting the ITU C&I Programme ITU maintains the C&I portal, which is used as an instrument for publishing the latest information related to the ITU C&I Programme. It represents all relevant information related to the ITU C&I Programme.

The ITU Secretariat shares the progress reports of ITU’s C&I Programme with relevant international bodies in the field of conformity assessment such as IEC, ISO, ILAC and IAF.

1.3 ITU-T activities related to interoperability events

In March 2015, ITU published a “whitelist” of mobile phones that were found to be compatible with Bluetooth-enabled vehicle-mounted hands-free terminals by an ITU test event.

Some ITU-T study groups started pilot projects on conformity assessment which aim is to develop test specifications for particular ICT technologies and organize relevant test events (e.g. OMCI-EPON, MNP, network management interface, etc.). The ICT devices which successfully pass the test may be added to the ITU Product Conformity Database.

ITU “Combating Counterfeit and Substandard ICT devices” event was organized in November 2014. At the conclusion of this event, ITU was invited to contribute by “using standards and C&I programs as a means to combat counterfeit and substandard ICT devices”. WTSA-16 created Resolution 96 (Hammamet, 2016) to strengthen the resolve of the membership to focus such activities within ITU-T SG11.

ITU-T study group identify topics for Interoperability events based on the market needs and suggestion from members to organize those events. Following their proposal, ITU conducts interoperability events at the request of ITU members. The following events were convened during the current study period:

– Joint APT/ITU Conformance and Interoperability event, (09-10 September 2013) (Bangkok, Thailand);

– Joint ITU/Continua Health Alliance Interoperability event on e-health, (Geneva, Switzerland, 28-31 October 2013);

– ITU test event on Performance assessment of vehicle-mounted mobile phones in conjunction with Hands-free Terminals according to Recommendations ITU-T P.1100 and ITU-T P.1110 (12-16 May 2014);

• This event found that only 30 per cent of mobile phones submitted for testing passed tests against performance requirements in Chapter 12 of Recommendations ITU-T P.1100 and ITU-T P.1110. As an outcome, the automotive industry urged ITU to publish a ‘whitelist’ of phones that are compliant with ITU-T P.1100/P.1110, in ITU’s conformity product database.

– 2nd joint APT/ITU Conformance and Interoperability event (Bangkok, Thailand, 25-26 August 2014);

– E-health testing and showcasing event (Geneva, ITU Headquarters, 10-12 February 2015);

– HATS Interoperability event on NGN supported by ITU and APT (Tokyo, Japan, 14-16 July 2015);

– 3rd joint APT/ITU Conformance and Interoperability event (Bangkok, Thailand, 7-8 September 2015);

– IPTV testing event (Geneva, Switzerland, 14-15 October 2015).

The complete list of the past C&I test events is available at http://www.itu.int/en/ITU-T/C-I/Pages/CIT-portal/archive-ITU-test-events.aspx.

1.4 In-force ITU-R Recommendations and Reports related to testing

The following lists have been extracted from the ITU-R website:

1.4.1 In-force ITU-R Recommendations related to testing

BO.600

Standardized set of test conditions and measurement procedures for the subjective and objective determination of protection ratios for television in the terrestrial broadcasting and the broadcasting-satellite services.

BS.645

Test signals and metering to be used on international sound programme connections.

BS.1657

Procedure for the performance test of automated audio identification systems.

BS.1693

Procedure for the performance test of automated query-by-humming systems.

BT.1210

Test materials to be used in assessment of picture quality.

BT.1729

Common 16:9 or 4:3 aspect ratio digital television reference test pattern.

F.1487

Testing of HF modems with bandwidths of up to about 12 kHz using ionospheric channel simulators.

M.1545

Measurement uncertainty as it applies to test limits for the terrestrial component of International Mobile Telecommunications-2000.

SM.1836

Test procedure for measuring the properties of the IF filter of radio monitoring receivers.

SM.1837

Test procedure for measuring the 3rd order intercept point (IP3) level of radio monitoring receivers.

SM.1838

Test procedure for measuring the noise figure of radio monitoring receivers.

SM.1839

Test procedure for measuring the scanning speed of radio monitoring receivers.

SM.1840

Test procedure for measuring the sensitivity of radio monitoring receivers using analogue-modulated signals.

SM.2060

Test procedure for measuring direction finder accuracy.

SM.2061

Test procedure for measuring direction finder immunity against multi-path propagation.

SM.2096

Test procedure for measuring direction finder sensitivity in the VHF/UHF frequency range.

1.4.2 In-force ITU-R Reports related to testing

BT.804

Definitions of parameters for automatic measurement of televison insertion test signals.

BT.1212

Measurements and test signals for digitally encoded colour television signals.

BT.1213

Test pictures and sequences for subjective assessments of digital codecs.

BT.2245

HDTV and UHDTV test materials for assessment of picture quality.

M.2032

Tests illustrating the compatibility between maritime radionavigation radars and emissions from radiolocation radars in the band 2 900-3 100 MHz.

M.2050

Test results illustrating the susceptibility of maritime radionavigation radars to emissions from digital communication and pulsed systems in the bands 2 900-3 100 MHz and 9 200-9 500 MHz.

M.2081

Test results illustrating compatibility between representative radionavigation systems and radiolocation and EESS systems in the band 8.5-10 GHz.

M.2115

Testing procedures for implementation of dynamic frequency selection.

M.2136

Theoretical analysis and testing results pertaining to the determination of relevant interference protection criteria of ground-based meteorological radars.

SM.2354

Alternative test procedure for measuring accuracy and immunity of direction finder using a simulator.

1.5 In-force ITU-T Recommendations and Supplements related to testing

E.424

Test calls.

E.439

Test call measurement to assess N-ISDN 64 kbit/s circuit-switched bearer service UDI in operation.

E.456

Test transaction for facsimile transmission performance.

E.300 series Suppl.5

Modelling of an experimental test design for the determination of inexperienced user difficulties in setting up international calls using nationally available instructions, or to compare different sets of instructions.

G.161.1

Do-no-harm testing.

G.650.1

Definitions and test methods for linear, deterministic attributes of single-mode fibre and cable.

G.650.2

Definitions and test methods for statistical and non-linear related attributes of single-mode fibre and cable.

G.650.3

Test methods for installed single-mode optical fibre cable links.

G.661

Definitions and test methods for the relevant generic parameters of optical amplifier devices and subsystems.

G.976

Test methods applicable to optical fibre submarine cable systems.

G.996.1

Test procedures for digital subscriber line (DSL) transceivers.

G.996.2

Single-ended line testing for digital subscriber lines (DSL).

G Suppl. 35

Guidelines concerning the measurement of wander.

G Suppl. 44

Test plan to verify B-PON interoperability.

G Suppl. 46

G-PON interoperability test plan between optical line terminations and optical network units.

H.264.1

Conformance specification for ITU-T H.264 advanced video coding.

H.265.1

Conformance specification for ITU-T H.265 high efficiency video coding.

H.810

Interoperability design guidelines for personal health systems.

H.811

Interoperability design guidelines for personal health systems: PAN/LAN/TAN interface.

H.812

Interoperability design guidelines for personal health systemsWAN interfaceCommon certified device class.

H.812.1

Interoperability design guidelines for personal health systemsWAN interfaceObservation upload certified device class.

H.812.2

Interoperability design guidelines for personal health systemsWAN interfaceQuestionnaires.

H.812.3

Interoperability design guidelines for personal health systemsWAN interfaceCapability exchange certified device class.

H.812.4

Interoperability design guidelines for personal health systemsWAN interfaceAuthenticated persistent session device class.

H.813

Interoperability design guidelines for personal health systemsHealth record network (HRN) interface.

H.821

Conformance of ITU-T H.810 personal health devices: Health record network (HRN) interface.

H.830.1

Conformance of ITU-T H.810 personal health devices: WAN interface Part 1: Web services interoperability: Sender.

H.830.2

Conformance of ITU-T H.810 personal health devices: WAN interface Part 2: Web services interoperability: Receiver.

H.830.3

Conformance of ITU-T H.810 personal health devices: WAN interface Part 3: SOAP/ATNA: Sender.

H.830.4

Conformance of ITU-T H.810 personal health devices: WAN interface Part 4: SOAP/ATNA: Receiver.

H.830.5

Conformance of ITU-T H.810 personal health devices: WAN interface Part 5: PCD-01 HL7 messages: Sender.

H.830.6

Conformance of ITU-T H.810 personal health devices: WAN interface Part 6: PCD-01 HL7 messages: Receiver.

H.830.7

Conformance of ITU-T H.810 personal health devices: WAN interface Part 7: Consent management: Sender.

H.830.8

Conformance of ITU-T H.810 personal health devices: WAN interface Part 8: Consent management: Receiver.

H.830.9

Conformance of ITU-T H.810 personal health devices: WAN interface Part 9: hData observation upload: Sender.

H.830.10

Conformance of ITU-T H.810 personal health devices: WAN interface Part 10: hData observation upload: Receiver.

H.830.11

Conformance of ITU-T H.810 personal health devices: WAN interface Part 11: Questionnaires: Sender.

H.830.12

Conformance of ITU-T H.810 personal health devices: WAN interface Part 12: Questionnaires: Receiver.

H.840

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN: USB host.

H.841

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 1: Optimized exchange protocol: Agent.

H.842

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 2: Optimized exchange protocol: Manager.

H.843

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 3: Continua Design Guidelines: Agent.

H.844

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 4: Continua Design Guidelines: Manager.

H.845.1

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5A: Weighing scales: Agent.

H.845.2

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5B: Glucose meter: Agent

H.845.3

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5C: Pulse oximeter: Agent

H.845.4

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5D: Blood pressure monitor: Agent.

H.845.5

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5E: Thermometer: Agent.

H.845.6

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5F: Cardiovascular fitness and activity monitor: Agent.

H.845.7

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5G: Strength fitness equipment: Agent.

H.845.8

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5H: Independent living activity hub: Agent.

H.845.9

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5I: Medication adherence monitor: Agent.

H.845.11

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5K: Peak expiratory flow monitor: Agent.

H.845.12

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5L: Body composition analyser: Agent.

H.845.13

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5M: Basic electrocardiograph: Agent.

H.845.14

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5N: International normalized ratio: Agent.

H.845.15

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 5O: Sleep apnoea breathing therapy equipment: Agent.

H.846

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 6: Device specializations: Manager.

H.847

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 7: Bluetooth low energy (BLE): Agent.

H.848

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 8: Bluetooth low energy (BLE): Manager.

H.849

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 9: Transcoding for Bluetooth low energy (BLE): Agent.

H.850

Conformance of ITU-T H.810 personal health devices: PAN/LAN/TAN interface Part 10: Transcoding for Bluetooth low e.

J.26

Test signals to be used on international sound-programme connections.

J.65

Standard test signal for conventional loading of a television channel.

J.67

Test signals and measurement techniques for transmission circuits carrying MAC/packet signals.

J.101

Measurement methods and test procedures for teletext signals.

J.133

Measurement of MPEG-2 transport streams in networks.

J.147

Objective picture quality measurement method by use of in-service test signals.

K.38

Radiated emission test procedure for physically large systems.

K.44

Resistibility tests for telecommunication equipment exposed to overvoltages and overcurrents – Basic Recommendation.

K.49

Test requirements and performance criteria for voice terminal telephones subject to disturbance. from digital mobile telecommunications.

K.54

Conducted immunity test method and level at fundamental power frequencies.

K.60

Emission levels and test methods for wireline telecommunication networks to minimize electromagnetic disturbance of radio se.

K.65

Overvoltage and overcurrent requirements for termination modules with contacts for test ports or surge protective devices

K.84

Test methods and guide against information leaks through unintentional electromagnetic emissions.

K.94

Mutual disturbance test method for evaluating performance degradation of converged terminal devices.

K.116

Electromagnetic compatibility requirements and test methods for radio telecommunication terminal equipment.

L.75

Test, acceptance and maintenance methods of copper subscriber pairs.

L.1005

Test suites for assessment of the universal charger solution.

L.1006

Test suites for assessment of the external universal power adapter solutions for stationary information and communication technology devices.

L.1007

Test suites for assessment of the External universal power adapter solutions for portable information and communication technology devices.

M.3170.4

Multi-technology network management: Conformance testing specification.

N.63

Test signals to be used by the broadcasting organizations during the preparatory period.

O.3

Climatic conditions and relevant tests for measuring equipment.

O.201

Q-factor test equipment to estimate the transmission performance of optical channels.

O.211

Test and measurement equipment to perform tests at the IP layer.

P.78

Subjective testing method for determination of loudness ratings in accordance with Recommendation P.76.

P.381

Technical requirements and test methods for the universal wired headset or headphone interface of digital mobile terminals.

P.382

Technical requirements and test methods for multi-microphone wired headset or headphone interfaces of digital wireless terminals.

P.564

Conformance testing for voice over IP transmission quality assessment models.

Q.921bis

Abstract test suite for LAPD conformance testing.

Q.933bis

Abstract test suite – Signalling specification for frame mode basic call control conformance testing for permanent virtual connections (PVCs).

Q.1600bis

Signalling system No. 7 – Interaction between ISDN user part ISUP '97 and INAP CS-1: Test suite structure and test purposes (TSS & TP).

Q.1912.5B

Interworking between session initiation protocol (SIP) and bearer independent call control protocol (BICC) or ISDN user part (ISUP): Protocol implementation conformance statement (PICS)

Q.1912.5C

Interworking between session initiation protocol (SIP) and bearer independent call control protocol (BICC) or ISDN user part (ISUP): Test suite structure and test purposes (TSS&TP) for profiles A and B.

Q.1912.5D

Interworking between session initiation protocol (SIP) and bearer independent call control protocol (BICC) or ISDN user part (ISUP): Test suite structure and test purposes (TSS&TP) for profile C.

Q.1912.5E

Interworking between session initiation protocol (SIP) and bearer independent call control protocol (BICC) or ISDN user part (ISUP): Abstract test suite (ATS) and partial protocol implementation extra information for testing (PIXIT) for profiles A and B.

Q.1912.5F

Interworking between session initiation protocol (SIP) and bearer independent call control protocol (BICC) or ISDN user part (ISUP): Abstract test suite (ATS) and partial protocol implementation extra information for testing (PIXIT) for profile C.

Q.2931B

Broadband integrated services digital network (B-ISDN) – Digital subscriber signalling system No. 2 (DSS2) – User-network interface (UNI) layer 3 specification for basic call/connection control: Protocol implementation conformance statement (PICS) proforma.

Q.2931C

Broadband integrated services digital network (B-ISDN) – Digital subscriber signalling system No. 2 (DSS2) – User-network interface (UNI) layer 3 specification for basic call/connection control: Test suite structure and test purposes (TSS & TP) for the user.

Q.2931D

Broadband integrated services digital network (B-ISDN) – Digital subscriber signalling system No. 2 (DSS2) – User-network interface (UNI) layer 3 specification for basic call/connection control: Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT) proforma for the user.

Q.2931E

Broadband integrated services digital network (B-ISDN) – Digital subscriber signalling system No. 2 (DSS2) – User-network interface (UNI) layer 3 specification for basic call/connection control: Test suite structure and test purposes (TSS & TP) for the network.

Q.2931F

Broadband integrated services digital network (B-ISDN) – Digital subscriber signalling system No. 2 (DSS2) – User-network interface (UNI) layer 3 specification for basic call/connection control: Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT) proforma for the network.

Q.3900

Methods of testing and model network architecture for NGN technical means testing as applied to public telecommunication networks.

Q.3901

Testing topology for networks and services based on NGN technical means.

Q.3902

Operational parameters to be monitored when implementing NGN technical means in public telecommunication networks.

Q.3903

Formalized presentation of testing results.

Q.3904

Testing principles for IMS model networks, and identification of relevant conformance, interoperability and functionality tests.

Q.3905

Conformance test plan for number portability requirements defined by ITU-T Q-Suppl.4.

Q.3906.1

Test scenarios and catalogue for testing fixed-broadband access networks using a model network – Part I.

Q.3909

The framework and overview of NGN conformance and interoperability testing.

Q.3910

Parameters for monitoring NGN protocols.

Q.3911

Parameters for monitoring voice services in NGN.

Q.3912

Set of parameters for monitoring next generation network streaming services.

Q.3913

Set of parameters for monitoring Internet of things devices.

Q.3920

Terms and definitions for conformance and interoperability.

Q.3925

Traffic flow types for testing quality of service parameters on model networks.

Q.3930

Performance testing of distributed systems - Concepts and terminology.

Q.3931.1

Performance benchmark for the PSTN/ISDN emulation subsystem of an IP multimedia system – Part 1: Core concepts.

Q.3931.2

Performance benchmark for the PSTN/ISDN emulation subsystem of an IP multimedia system – Part 2: Subsystem configurations and benchmarks.

Q.3931.3

Performance benchmark for the PSTN/ISDN emulation subsystem of an IP multimedia system – Part 3: Traffic sets and traffic profiles.

Q.3931.4

Performance benchmark for the PSTN/ISDN emulation subsystem of an IP multimedia system – Part 4: Reference load network quality parameters.

Q.3932.1

IMS/NGN performance benchmark – Part 1: Core concept .

Q.3932.2

IMS/NGN performance benchmark – Part 2: Subsystem configurations and benchmarks.

Q.3932.3

IMS/NGN performance benchmark – Part 3: Traffic sets and traffic profiles.

Q.3932.4

IMS/NGN performance benchmark – Part 4: Testing of the performance design objectives.

Q.3933

Reference benchmarking, background traffic profiles and KPIs for VoIP and FoIP in fixed networks.

Q.3940

NGN/IMS interconnection tests between network operators at the IMS ‘Ic’ interface and NGN NNI / SIP-I.

Q.3941.1

Network integration testing between SIP and ISDN/PSTN network signalling protocols – Part 1: Test suite structure and test purposes for SIP-ISDN.

Q.3941.2

Network integration testing between SIP and ISDN/PSTN network signalling protocols – Part 2: Abstract test suite and partial protocol implementation extra information for testing proforma specification for SIP-ISDN.

Q.3941.3

Network integration testing between SIP and ISDN/PSTN network signalling protocols – Part 3: Test suite structure and test purposes for SIP-SIP.

Q.3941.4

Network integration testing between SIP and ISDN/PSTN network signalling protocols – Part 4: Abstract test suite and partial protocol implementation extra information for testing proforma specification for SIP-SIP.

Q.3941.5

Network integration testing between SIP and ISDN/PSTN network signalling protocols – Part 5: TSS&TP for network integration tests between ISDN-ISDN and ISDN-PSTN over SIP-II NNI / SIP-I NNI.

Q.3942.1

Conformance test specification for the terminating identification restriction using IP multimedia core network subsystem – Part 1: Protocol implementation conformance statement.

Q.3942.2

Conformance test specification for the terminating identification restriction using IP multimedia core network subsystem – Part 2: Network side; Test suite structure and test purposes.

Q.3942.3

Conformance test specification for the terminating identification restriction using IP multimedia core network subsystem – Part 3: User side; Test suite structure and test purposes.

Q.3945

Test specifications for next generation network services on model networks - Test set 1.

Q.3946.1

Conformance tests specification for the session initiation protocol – Part 1: Protocol implementation conformance statement proforma.

Q.3946.2

Conformance tests specification for the session initiation protocol – Part 2: Test suite structure and test purposes.

Q.3946.3

Conformance tests specification for the session initiation protocol - Part 3: Abstract test suite and partial protocol implementation extra information for testing (PIXIT) proforma.

Q.3948

Service testing framework for VoIP at the user-to-network interface of next generation networks.

Q.3949

Real-time multimedia service testing framework at the user-to-network interface of next generation networks.

Q.3950

Testing and model network architecture for tag-based identification systems and functions.

Q.3951

Real-time Internet Protocol based on the ITU-T T.38 supporting facsimile service testing framework at the user-to-network interface of next generation networks.

Q.3960

Framework of Internet related performance measurements.

Q.Suppl 1

Signalling System No.7 testing and planning tools.

R.51

Standardized text for distortion testing of the code-independent elements of a complete circuit.

R.51bis

Standardized text for testing the elements of a complete circuit.

T.5

Test methodology for Group 3 facsimile processing equipment in the Public Switched Telephone Network.

T.22

Standardized test charts for document facsimile transmissions.

T.23

Standardized colour test chart for document facsimile transmissions.

T.24

Standardized digitized image set.

T.803

Information technology – JPEG 2000 image coding system: Conformance testing.

T.834

Information technology – JPEG XR image coding system – Conformance testing.

T.Suppl 1

Conformance testing requirements for Recommendations of the T.170-series.

V.56ter

Test procedure for evaluation of 2-wire 4 kHz voiceband duplex modems.

X.245

Information technology – Open Systems Interconnection – Connection-oriented Session protocol: Protocol Implementation Conformance Statement (PICS) proforma.

X.246

Information technology – Open Systems Interconnection – Connection-oriented Presentation protocol: Protocol Implementation Conformance Statement (PICS) proforma.

X.247

Information technology – Open Systems Interconnection – Protocol specification for the Association Control Service Element: Protocol Implementation Conformance Statement (PICS) proforma.

X.248

Information technology – Open Systems Interconnection – Reliable Transfer: Protocol Implementation Conformance Statement (PICS) proforma.

X.249

Information technology – Open Systems Interconnection – Remote Operations: Protocol Implementation Conformance Statement (PICS) proforma.

X.255

Information technology – Open Systems Interconnection – Connectionless Session protocol: Protocol Implementation Conformance Statement (PICS) proforma.

X.256

Information technology – Open Systems Interconnection – Connectionless Presentation protocol: Protocol Implementation Conformance Statement (PICS) proforma.

X.257

Information technology – Open Systems Interconnection – Connectionless protocol for the Association Control Service Element: Protocol Implementation Conformance Statement (PICS) proforma.

X.290

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – General concepts.

X.291

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – Abstract test suite specification.

X.292

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – The Tree and Tabular Combined Notation (TTCN).

X.293

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – Test realization.

X.294

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – Requirements on test laboratories and clients for the conformance assessment process.

X.295

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – Protocol profile test specification.

X.296

OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications – Implementation conformance statements.

X.481

Message handling systems – P2 protocol PICS proforma.

X.482

Message handling systems – P1 Protocol PICS proforma.

X.483

Message handling systems – P3 Protocol PICS proforma.

X.484

Message handling systems – P7 protocol PICS proforma.

X.485

Message handling systemsVoice messaging system Protocol Implementation Conformance Statement (PICS) proforma.

X.486

Message handling systems – Pedi protocol PICS proforma.

X.487

Message handling systems – IPM-MS attributes PICS proforma.

X.488

Message handling systems – EDI-MS attributes PICS proforma.

Annex 3: References

– Dubai Action Plan;

– Plenipotentiary Resolution 177 (Dubai, 2014);

– WTDC Resolution 47 (Dubai, 2014);

– WTDC Resolution 77 (Dubai, 2014);

– ITU-D Study Group Question 4/2: http://itu.ing/go/CI_Question4_2;

– ITU C&I programme: http://itu.int/go/CI_Development; contact: c&i@gmail.com;

– ITU Guidelines: http://itu.int/go/CI_Guidelines;

– ISO/CASCO: http://www.iso.org/iso/home/about/conformity-assessment/casco.htm;

– WTO-TBT: https://www.wto.org/english/tratop_e/tbt_e/tbt_e.htm.

_______________

[35] Recommendation ITU-T K.44, “Resistibility tests for telecommunication equipment exposed to overvoltages and overcurrents –  Basic Recommendation”.

[36] Recommendation ITU-T L.1005, “Test suites for assessment of the universal charger solution”.

[37] Recommendation ITU-T P.564, “Conformance testing for voice over IP transmission quality assessment models”.