OpenSolaris Binary Distro Test Automation Coverage

Basic Information

This page is providing basic info about test automation coverage for G11n OpenSolaris Binary Distro testing including brief description of used tools and technologies

Test Harness, Tools, Technologies

Test Harness:

• OpenSolaris/Solaris G11n Automation - CTI for TET
• OpenSolaris Desktop Automation - QAAE/XNEE

Tools:

• Dogtail (Desktop GUI)
• expect/sunexpect (CLI)

Technologies:

• Scripting - ksh, perl
• Programming languages - C, Java

Automation Coverage

You can find below brief automated tests description for particular test areas

Install and Upgrade

Already automated:

• installation errors checking
• duplicate files in packages
• pgkchk (broken symlinks, correct file permissions, file dates ..)
• default locale setting - system default locale is set according user's installation setting
• core files - looking for core files after installation
• localeadm test - testing installation/uninstallation of locales using localeadm tool

Can be automated:

• dependency check (perl script is available, 5hrs)

Can not be automated:

• ...

Xserver - i18n

Already automated:

• N/A

Can be automated:

• automatted tools could be used by X team x11perf and VSW5:
  **For x11perf issue, X team are fixing it.
  **For VSW5 issue, I will contact with OpenGroup community later.

Can not be automated:

• ...

Keyboards - xkb data

Already automated:

• xkb maps validation - cheking if xserver keyboard maps does not have syntax errors and can be compiled properly

Can be automated:

• ...

Can not be automated:

• ...

Input Methods (ibus, iiimf, scim)

Already automated:

• iiimf emea input - dogtail/phyton based script testing iiim keyboard emulation and comparing with golden data if all characters are inputed correctly and according particular layout specifications

Can be automated:

• basic input test (language switch, a short text input) by utilizing the above dogtail/phyton based script or any other tool.

Can not be automated:

• ...

Fonts

Already automated:

• ...

Can be automated:

• ...

Can not be automated:

• ...

Locales

Already automated:

• locale setting - it's possible to set every locale in console/terminal enviroment
• login/logout - it is possible to log in OpenSolaris Desktop session for every locale
• symlinks
• locale-regression-check - checking for regression of previous issues
• validate locale - locale data validation (date formats, currency, numeric formats ..)
• sorting - checking if sorting in particular locales works correctly
• binary cat - checking if all needed 64bit and 32bit binary files for all locales are installed on the system
• x-files - checking for missing UTF-8 localedef extensions file
• Locale map test - check whether entries in /usr/lib/locale/xx_XX.XXXXXXXX/locale_map (for all locales) dont have C set as a locale (LANG and LC_MESSAGES variables are checked whether they arent assigned "C")

Can be automated:

• new locales testing
• updates of locale data (automate the test "golden images" update)

Can not be automated:

• nothing

Desktop

Already automated:

• Accessories – Archive Manager/Calculator/Charactor Map/Gedit/PDA Sync
• Sound & Video – Sound Juicer/RealPlay/Totem/Gnome Sound Recorder/Gnome Volume Control
• Graphics – Gimp/Gthumb/Gtkam/Eog/Gnome Screen Shot
• Internet – Firefox/Thunderbird/Evolution/Pidgin/Ekiga
• System Tools – Gnome System Log/Fsexam/Gnome Terminal/Dtpower/Baobab/Gnome About/nmapfe/Gnome System Monitor/sdtsmartcardadmin.
• Office – Evince/StarSuite
• Developer Tools – Glade3/Ghex2/devhelp/Developer Guide
• Other – Yelp/gnome-search-tool

Note:

1. Only sanity checks are done by automation. Most cases are just launch the application and capture the screen shots of the main GUI/Help/About.
2. Some applications are to be obsoleted, such as sdtsmartcardadmin.

Can be automated:

Can not be automated:

Localization

Already automated:

• symlinks presence - all locales contains correct symlinks and point to dirrectories with localization .mo files
• mo check - checking .mo files (l10n messages) are present on the system for supported languages and application
• l10n check - testing localization of following utilities: bootadm, idmap, routeadm, hddlocator, dscp, CIFS, KMF, ucodeadm, dladm, stmsboot, ipseckey, sharemgr, duckhorn, localecrt, pmconfig, NDMP, pmconfig, zfs, fwflash
• msgutil script has been available to check format error. Any other light weight scripts are welcome.

Can be automated:

• Technical check of the file format. This check should be executed before integration into CWS in scope of PIT or PCT for source files (e.g. po)
  • Run GNU gmsgfmt and check the file structure (% gmsgfmt --check --verbose translated_file.po) [1 day]
  • Run parameters check (e.g. {1} or %d) [2 days]
  • Run HTML tags check in comparison with English file [2 days]
  • Check the existence of "#, fuzzy" in .po files (due to CR 6722603) [1 day]
    (see http://www.opensolaris.org/os/community/int_localization/software_translation/common_mistakes/ for additional possibilities for technical checks)
• Automated new word count check by leverage through WorldServer.

Can not be automated:

• Linguistic check of translations

ON foundation ( libc, iconv .. )

Already automated:

• ...

Can be automated:

• ...

Can not be automated:

• ...