�1���� Scope����������
�2���� References
�3���� Definitions
������� 3.1���� Terms defined elsewhere�������
������� 3.2���� Terms defined in this Recommendation�����������
�4���� Abbreviations and acronyms�������������
�5���� Conventions������������
�6���� General recommendations concerning the subjective quality evaluation of
multiparty telemeetings����
�7���� Multiparty specific aspects in subjective quality evaluation�
�8���� Guidance to suitable test methods�
������� 8.1���� Test method decision criteria
������� 8.2���� Flow charts to be used when selecting test methods��
Annex A � Set-up of a multiparty telemeeting assessment test����
������� A.1���� Assessment of conversational quality � Conversation tests������
������� A.2���� Assessment of non-interactive quality � Non-interactive tests
Annex B � Assessment of telemeetings with text-based communication and
graphical information means (e.g., web conferencing)����
Annex C � Assessment of video-only telemeetings����
Annex D � Effect of transmission delays on telemeeting quality����
������� D.1���� Background��
������� D.2���� Existing test task recommendations��
������� D.3���� Recommended test tasks�������
������� D.4���� Set-up of a delay test
������� D.5���� Test subjects
������� D.6���� Training session����������
� ������D.7���� Instructions��
������� D.8���� Test questions������������
������� D.9���� Objective measurements�������
������ D.10���� Effects of delay����������
Annex E � Assessment of spatial audio and 3D video reproduction� of multiparty
telemeetings����
������� E.1���� General remarks���������
������� E.2���� Recommendations concerning spatial audio���
������� E.3���� Remarks concerning 3D video
Annex F � Assessment of asymmetric multiparty telemeetings����
������� F.1���� Problem statement���
������� F.2���� Translation of technical degradations into possible perceptual
impairments�����
������� F.3���� Influencing factors�����
������� F.4���� Experiment design�����
������� F.5���� Rating scales�
������� F.6���� Test stimuli����
������� F.7���� Data analysis
Annex G � Assessment of multiparty telemeetings with non-stationary quality����
Annex H � Assessment of multiparty telemeetings using multi-dimensional scaling
methods����
Appendix I � Influential factors����
Appendix II � Overview of multiparty non-interactive test stimuli and
conversation test tasks����
������ II.1���� Non-interactive audio-only stimuli:������
������ II.2���� Non-interactive video-only stimuli�������
������ II.3���� Non-interactive audiovisual stimuli������
������ II.4���� Audio-only conversation tasks
������ II.5���� Audiovisual conversation tasks��������������
Appendix III � Examples of multiparty conversation test tasks (audio-only and
audiovisual):� Free conversation����
Appendix IV � Examples of multiparty conversation test tasks (audio-only):�
Three-party conversation test scenarios 3CTs����
������ IV.1���� Introduction�
������ IV.2���� Test scenario development���
������ IV.3���� Scenario validation����
������ IV.4���� Cultural aspects����������
Appendix V � Examples of multiparty conversation test tasks (audiovisual):�
Audiovisual multi-point tasks for three parties� (Survival task, Leavitt task,
Brainstorming task)����
������� V.1���� Overview and most suitable task���������
������� V.2���� Leavitt task���
������� V.3���� Brainstorming task����
������� V.4���� Survival task�
Appendix VI � Additional proposals for multiparty conversation test tasks
(audiovisual):�� Extended survival tasks scenarios and celebrity name guessing
task����
������ VI.1���� Overview and background�����
������ VI.2���� Modification of survival scenarios from Appendix V����
������ VI.3���� Three additional survival scenarios�����
������ VI.4���� Celebrity name guessing task
Appendix VII � Additional proposals for multiparty conversation test tasks
(audiovisual):�� Formal and informal multiparty video conferences����
����� Page
Appendix VIII � Overview of documents describing suitable test methods����
���� VIII.1���� Baseline test methods on which the current Recommendation is
based�������������
���� VIII.2���� Multiparty specific recommendations to adapt the baseline test
methods��������
���� VIII.3���� Further test methods, which are referred to in the annexes of
this Recommendation����
Bibliography����