What are the various kinds of Servers (Web Servers, Application Servers, etc.)? [Source jguru.com, author Paul Danckaert] / Difference between a Web Server and Application Server.

The servers involved in handling and processing a user's request break down into a few basic types, each of which may have one or more tasks it solves. This flexibility gives developers a great deal of power over how applications will be created and deployed, but also leads to confusion over what server is able to, or should, perform a specific task.

Starting at the basic level, a user is typically submitting a request to a system through a web browser. (We are conveniently ignoring all other types of clients (RMI, CORBA, COM/DCOM, Custom, etc..) for the time being for purposes of clarity.) The web request must be received by a Web Server (otherwise known as an HTTP Server) of some sort. This web server must handle standard HTTP requests and responses, typically returning HTML to the calling user. Code that executes within the server environment may be CGI driven, Servlets, ASP, or some other server-side programming language, but the end result is that the web server will pass back HTML to the user.

The web server may need to execute an application in response to the users request. It may be generating a list of news items, or handling a form submission to a guest book. If the server application is written as a Java Servlet, it will need a place to execute, and this place is typically called a Servlet Engine. Depending on the web server, this engine may be internal, external, or a completely different product. This engine is continually running, unlike a traditional CGI environment where a CGI script is started upon each request to the server. This persistance gives a servlet connection and thread pooling, as well as an easy way to maintain state between each HTTP request. JSP pages are usually tied in with the servlet engine, and would execute within the same space/application as the servlets.

There are many products that handle the web serving and the servlet engine in different manners. Netscape/iPlanet Enterprise Server builds the servlet engine directly into the web server and runs within the same process space. Apache requires that a servlet engine run in an external process, and will communicate to the engine via TCP/IP sockets. Other servers, such as MS IIS don't officially support servlets, and require add-on products to add that capability.

When you move on to Enterprise JavaBeans (and other J2EE components like JMS and CORBA) you move into the application server space. An Application Server is any server that supplies additional functionality related to enterprise computing -- for instance, load balancing, database access classes, transaction processing, messaging, and so on.

EJB Application Servers provide an EJB container, which is the environment that beans will execute in, and this container will manage transactions, thread pools, and other issues as necessary. These application servers are usually stand-alone products, and developers would tie their servlets/JSP pages to the EJB components via remote object access APIs. Depending on the application server, programmers may use CORBA or RMI to talk to their beans, but the baseline standard is to use JNDI to locate and create EJB references as necessary.

Now, one thing that confuses the issue is that many application server providers include some or all of these components in their product. If you look at WebLogic (http://www.beasys.com/) you will find that WebLogic contains a web server, servlet engine, JSP processor, JMS facility, as well as an EJB container. Theoretically a product like this could be used to handle all aspects of site development. In practice, you would most likely use this type of product to manage/serve EJB instances, while dedicated web servers handle the specific HTTP requests.

Types of tests to be included in a System Test (source Wikipedia)

1. Graphical user interface testing
2. Usability testing
3. Software performance testing
4. Compatibility testing
5. Exception handling
6. Load testing
7. Volume testing
8. Stress testing
9. Security testing
10. Scalability testing
11. Sanity testing
12. Smoke testing
13. Exploratory testing
14. Ad hoc testing
15. Regression testing
16. Installation testing
17. Maintenance testing
18. Recovery testing and failover testing.
19. Accessibility testing, including compliance with certain compliance needed for the disabled.

Six major components of test automation framework (Softwaregenius.com)

A test automation infrastructure, or framework, consists of test tools, equipment, test scripts, procedures, and people needed to make test automation efficient and effective. The creation and maintenance of a test automation framework are key to the success of any test automation project within an organization.

The implementation of an automation framework generally requires an automation test group. The responsibility of this group is to develop test automation infrastructure, test libraries, and tests tools.


The idea behind an automation infrastructure is to ensure the following:a) Different test tools and equipment are coordinated to work together.
b) The library of the existing test case scripts can be reused for different test projects, thus minimizing the duplication of development effort.
c) Nobody creates test scripts in their own ways.
d) Consistency is maintained across test scripts.
e) The test suite automation process is coordinated such that it is available just in time for regression testing.
f) People understand their responsibilities in automated testing.


Components of a typical test automation framework are as described in the following figure.


1) System to Be Tested: This is the first component of an automation infrastructure. The subsystems of the system to be tested must be stable; otherwise test automation will not be cost effective. All the subsystems must be stable and work together as a whole before the start of an automation test project.


2) Testing Platform: The testing platform and facilities, that is, the network setup on which the system will be tested, must be in place to carry out the test automation project. For example, a procedure to download the image of the SUT, configuration management utilities, servers, clients, routers, switches, and hubs are necessary to set up the automation environment to execute the test scripts.


3) Test Case Library: It is useful to compile libraries of reusable test steps of basic utilities to be used as the building blocks of automated test scripts. Each utility typically performs a distinct task to assist the automation of test cases. Examples of such utilities are ssh (secure shell) from client to server, exit from client to server, response capture, information extraction, rules for verdicts, verdict logging, error logging, cleanup, and setup.


4) Automated Testing Practices: The procedures describing how to automate test cases using test tools and test case libraries must be documented. A template of an automated test case is useful in order to have consistency across all the automated test cases developed by different engineers. A list of all the utilities and guidelines for using them will enable us to have better efficiency in test automation. In addition, the maintenance procedure for the library must be documented.


5) Testing Tools: Different types of tools are required for the development of test scripts. Examples of such tools are test automation tool, traffic generation tool, traffic monitoring tool, and support tool. The support tools include test factory, requirement analysis, defect tracking, and configuration management tools. Integration of test automation and support tools, such as defect tracking, is crucial for the automatic reporting of defects for failed test cases. Similarly, the test factory tool can generate automated test execution trends and result patterns.


6) Test Administrator: The automation framework administrator does the following

a) Manages test case libraries, test platforms, and test tools;
b) Maintains the inventory of templates;
c) Provides tutorials; and
d) Helps test engineers in writing test scripts using the test case libraries.
e) Provides tutorial assistance to the users of test tools and maintains a liaison with the tool vendors and the users.


Source: http://www.softwaretestinggenius.com/articalDetails.php?qry=957

Configuration Management Audit...

There are two meanings for the project management process of configuration management.

1. It can be used for the process of identifying, tracking, and managing of all the physical assets of a project. The items that you track under configuration management are called “configuration items” in the Capability Maturity Model (CMMI). 
2. It can also refer to the process of identifying, tracking and managing of all the characteristics of the assets of a project. These characteristics can also be referred to as product “metadata.” This is closer to the definition of configuration management in the Project Management Body of Knowledge (PMBOK®) from the Project Management Institute.

The following model describes the five major aspects of configuration management.

Planning. You need to plan ahead to create the processes, procedures, tools, files, and databases for managing the project assets or the metadata. You also may need to gain an agreement on exactly what assets are important, how you will define them, how they will be categorized, classified, numbered, reported, etc. The results of this up-front planning are documented in a Configuration Management Plan.

Part of your planning process should be to assign configuration tracking numbers to each type of configuration item.

Tracking. It’s important to understand the baseline for all configuration items. In other words, for each configuration item, you need to understand what you have at the beginning of the project. In many cases, you may have nothing to start with. In other cases, like physical assets, you may have some assets to begin with. The purpose of your tracking processes is to ensure that you can track all changes to a configuration item throughout the project.

You need processes and systems designed to identify when assets are assigned to your project, where they go, what becomes of them, who is responsible for them and how they’re disposed of. Since a project has a beginning and end, ultimately all the assets need to go somewhere. This could be in a final deliverable, into the operations/support area, scrapped, etc. You should be able to dissect each major deliverable of the project and show where all the pieces and parts came from, and where they reside after the project ends.

Managing. Managing assets means ensuring that they’re secure, protected, and used for the right purposes. For example, it doesn’t do any good to track purchased assets that your project doesn’t need in the first place. Also, your tracking system may show expensive components sitting in an unsecured storage room, but is that really the proper place for them? Managing assets has to do with acquiring what you need and only what you need. You also have to make sure you have the right assets at the right place at the right time.

Reporting. You need to be able to report on the project assets, usually in terms of what you have and where they are, as well as financial reporting that can show cost, budget, depreciation, etc.

Auditing. Auditing involves validating that the actual configuration elements (whatever they are) at any given time are the same as what you expect. Many projects get in trouble when they start to lose track of physical assets (for instance, material, supplies, code or other configuration items) or if the physical characteristics (metadata) of your deliverables is different that what you expect.

The auditing process is used to validate that the configuration elements match up with your expectations. These expectations are based on the original baseline, plus any change requests that you have processed up to the current time.

Source: http://blogs.techrepublic.com.com/tech-manager/?p=370&tag=content;leftCol

Stat tools - when to use what

1. Box Plots: used to tell if the data set is skewed by looking at the relative positions of the median, quartiles and the tail. Easy to spot outliers.
2. Scatter Plots: when we are interested in the relationships between two attributes. Offer visual assessment.
3. Control Charts: to know whether the data is within acceptable bounds.
4. Correlation: measure the correlation between two variables. Measure the strength of relationship / association.
5. Linear Regression: express relation as a linear formula.
6. Multivariate Regression: investigate relationship between one dependent variable and two or more independent variables.