Planet SoC -

SoC 2008 Beyond the End

csfalcon — Mon, 08 Sep 2008 22:33:52 +0200

Google Summer of Code 2008 officially ended on August 18th. Since that deadline, I have been working to solve a few non-functional problems I discovered during testing stage.

By week 11, Skoll Client was collecting runtime information for each of the MySQL tests. During the development stage, I only tested this feature with a few MySQL tests. However, during testing stage I discovered this feature would take way too much time and resources for majority of the users. To completely test one configuration of MySQL, it took Skoll Client over 10 hours because MySQL server must be restarted after each test. Also, the runtime information collected was over 3 GB (zipped!). If the user is only interested in the coverage of the MySQL test suite, the per-test runtime information is a major overkill. The solution was to have two runtime information collection flags in the Skoll Client, —gcov and —detailed_gcov, to collect runtime information for all tests and runtime information per test. I doubt any user besides me would want to use the —detailed_gcov option.
Generating runtime information reports from the database has always been slow due to the large amount of data collected for MySQL. Now that the database structure and the SQL queries is more concrete, I can finally spend some time optimize for performance. With a few tweaks to the SQL and a few added indices, I was able to decrease the runtime of some queries from over 2 minutes down to 0.04 seconds. I am pretty new to optimizing database performance, learning about the ?explain? command really helped me understand what was going on with a query. However, even with this huge performance improvement, certain types of reports such as ?per source line? is still too slow for data collected for multiple configurations. The visualization techniques I am using is not mature enough to handle large software system like MySQL just yet.

My plans for this project after Summer of Code is as follows:

Skoll Client is feature complete, I have no plans to add new features in the near future. Of course, bug fixes and patches will be added.
The visualization techniques I worked on look promising, but using them on MySQL right now is costly in both time and computing resources. I will continue to research ways to visualize multi-configuration runtime data, but on smaller software systems.
For future updates on the Skoll Project, visit http://www.cs.umd.edu/projects/skoll/contribute/mysql.html.

SoC 2008 Beyond the End

csfalcon — Mon, 08 Sep 2008 22:33:52 +0200

Google Summer of Code 2008 officially ended on August 18th. Since that deadline, I have been working to solve a few non-functional problems I discovered during testing stage.

By week 11, Skoll Client was collecting runtime information for each of the MySQL tests. During the development stage, I only tested this feature with a few MySQL tests. However, during testing stage I discovered this feature would take way too much time and resources for majority of the users. To completely test one configuration of MySQL, it took Skoll Client over 10 hours because MySQL server must be restarted after each test. Also, the runtime information collected was over 3 GB (zipped!). If the user is only interested in the coverage of the MySQL test suite, the per-test runtime information is a major overkill. The solution was to have two runtime information collection flags in the Skoll Client, —gcov and —detailed_gcov, to collect runtime information for all tests and runtime information per test. I doubt any user besides me would want to use the —detailed_gcov option.
Generating runtime information reports from the database has always been slow due to the large amount of data collected for MySQL. Now that the database structure and the SQL queries is more concrete, I can finally spend some time optimize for performance. With a few tweaks to the SQL and a few added indices, I was able to decrease the runtime of some queries from over 2 minutes down to 0.04 seconds. I am pretty new to optimizing database performance, learning about the ?explain? command really helped me understand what was going on with a query. However, even with this huge performance improvement, certain types of reports such as “per source line” is still too slow for data collected for multiple configurations. The visualization techniques I am using is not mature enough to handle large software system like MySQL just yet.

My plans for this project after Summer of Code is as follows:

Skoll Client is feature complete, I have no plans to add new features in the near future. Of course, bug fixes and patches will be added.
The visualization techniques I worked on look promising, but using them on MySQL right now is costly in both time and computing resources. I will continue to research ways to visualize multi-configuration runtime data, but on smaller software systems.
For future updates on the Skoll Project, visit http://www.cs.umd.edu/projects/skoll/contribute/mysql.html.

Week 12 - A Test Scheduler for the MySQL Build Farm Initiative

csfalcon — Tue, 19 Aug 2008 05:04:37 +0200

KEY ACCOMPLISHMENTS LAST WEEK

Refactored and cleaned up source code.
Updated JavaDoc.
Uploaded source code to code.google.com.

KEY TASKS THAT STALLED LAST WEEK

Tried to upload source code to launchpad, but code.google.com was easier to use.

KEY CONCERNS

None

TASKS IN THE UPCOMING WEEK

Keep working on the project until the semester starts (Sept 2nd), at least.
Integrate all server side processing scripts and tools (Java, Perl, R, HTML).
Collect and analyze MySQL runtime information.

Final GSoC revision of SIGAR I_S plugin

m_ch — Sat, 16 Aug 2008 20:35:00 +0200

The GSoC is about to end up and it is time to sum up what has been done within the last weeks.

News

*Error handling added. Plugin uses the following mechanism to gather and report errors. When we can’t load/open SIGAR library or get any information from it we pass the error to user informing about that. On the other hand when only some information is not available we rather print warning about that to MySQL .err file.
Also we check for return value for each schema_store_record function.

*Plugin has been merged into the MySQL 6 server code. Now you can download and compile the MySQL server with SIGAR plugin already being there.

Downloads

Standalone SIGAR plugin is available at lp:~m.ch/mysql-server/sigar-plugin.
SIGAR plugin merged into the MySQL server is available at lp:~m.ch/mysql-server/mysql-6.0-sigar-plugin.

Updated installation guideline and documentation is available in README file.

I hope you will like it and enjoy using it.

Running Shell Commands in Java

csfalcon — Tue, 12 Aug 2008 06:54:21 +0200

The Skoll Client retrieves a set of commands from the Skoll Server to compile and test MySQL; these commands are UNIX shell commands. My Google Summer of Code project is to work on the Java Skoll Client, I spent a great deal of time and effort getting these shell commands to run well in Java. Running shell commands in Java is not always straightforward, here are some techniques I learned to get the job done.

Use Java’s ProcessBuilder class instead of Runtime class for Java 5.0+ projects. There are two reasons. First, ProcessBuilder has redirectErrorStream() method that automatically merges the standard output and standard error together so the output of a command would make more sense and more importantly the process won’t freeze if the developer doesn’t clear the error output stream. Second, the environment() method returns a nice Map the developer can use to manipulate the environment in which the command should run. Unfortunately, I must support Java 1.4 and 5.0 so I had to do everything the hard way.

Modifying the PATH environment variable does not add a location for Java to locate executables, at least not the way I expected it to. I was trying to run the “bzr” command which is not in my default path, I thought setting the PATH environment variable to include the non-default path would allow Java to find “bzr”. That did not work. The environment settings do not take affect until the process has started, but Java cannot start the process because it cannot find the “bzr” command. For now, I set the PATH in the shell script that starts the Skoll Client. But one possible solution is to set the environment variables, then run the command “sh -c ‘bzr’” (not yet tested).

Passing a shell command to Java does not always work. The way Java parses the commands does unexpected things, at least to me at first. I found that parsing the commands myself then passing them to Java guarantees correct execution. This command parsing algorithm divides one command string into an array of command parts; the command parts are divided by whitespace, unless the whitespace appears in quotation. Here is a simple method that prepares the command for Java.

private String[] prepareCommand(String command)
{
   ArrayList commandParts = new ArrayList();
   StringBuffer currPart = new StringBuffer();
   boolean inQuote = false;

   for (int x = 0; x < command.length(); x++)
   {
      char ch = command.charAt(x);

      if (ch == ''' || ch == '"')
      {
         inQuote = !inQuote;
      }
      else if (Character.isWhitespace(ch) && !inQuote)
      {
         commandParts.add(currPart.toString());
         currPart = new StringBuffer();
      }
      else
      {
         currPart.append(ch);
      }

      // get the last part of the command
      if (x == command.length() - 1)
      {
         commandParts.add(currPart.toString());
      }
   }

   String[] coms = new String[commandParts.size()];
   commandParts.toArray(coms);

   return coms;
}

Running Shell Commands in Java

csfalcon — Tue, 12 Aug 2008 06:54:21 +0200

private String[] prepareCommand(String command)
{
   ArrayList commandParts = new ArrayList();
   StringBuffer currPart = new StringBuffer();
   boolean inQuote = false;

   for (int x = 0; x < command.length(); x++)
   {
      char ch = command.charAt(x);

  if (ch == ''' || ch == '"')
  {
     inQuote = !inQuote;
  }
  else if (Character.isWhitespace(ch) &amp;&amp; !inQuote)
  {
     commandParts.add(currPart.toString());
     currPart = new StringBuffer();
  }
  else
  {
     currPart.append(ch);
  }

  // get the last part of the command
  if (x == command.length() - 1)
  {
     commandParts.add(currPart.toString());
  }


}

String[] coms = new String[commandParts.size()];
   commandParts.toArray(coms);

return coms;
}

For more information and source code related to running shell commands in Java, see ShellCommandsWithJava.

Running Shell Commands in Java

csfalcon — Tue, 12 Aug 2008 06:54:21 +0200

private String[] prepareCommand(String command)
{
   ArrayList commandParts = new ArrayList();
   StringBuffer currPart = new StringBuffer();
   boolean inQuote = false;

   for (int x = 0; x < command.length(); x++)
   {
      char ch = command.charAt(x);

  if (ch == ''' || ch == '"')
  {
     inQuote = !inQuote;
  }
  else if (Character.isWhitespace(ch) &amp;&amp; !inQuote)
  {
     commandParts.add(currPart.toString());
     currPart = new StringBuffer();
  }
  else
  {
     currPart.append(ch);
  }

  // get the last part of the command
  if (x == command.length() - 1)
  {
     commandParts.add(currPart.toString());
  }


}

String[] coms = new String[commandParts.size()];
   commandParts.toArray(coms);

return coms;
}

For more information and source code related to running shell commands in Java, see ShellCommandsWithJava.

Running Shell Commands in Java

csfalcon — Tue, 12 Aug 2008 06:54:21 +0200

Error while rendering page:null

Week 11 - A Test Scheduler for the MySQL Build Farm Initiative

csfalcon — Tue, 12 Aug 2008 06:47:57 +0200

KEY ACCOMPLISHMENTS LAST WEEK

Finished per test runtime information collection modifications.
Collected and analyzed per test runtime information for a few suspicious configurations and tests.
Refactored and cleaned up source code.

KEY TASKS THAT STALLED LAST WEEK

None

KEY CONCERNS

None

TASKS IN THE UPCOMING WEEK

Refactor and clean up source code.
Update JavaDoc.

Collect Runtime Information for Each MySQL Test

csfalcon — Tue, 05 Aug 2008 06:34:03 +0200

In a previous post, I talked about the method Skoll used to collect MySQL runtime information for non-default configurations. At the time, the runtime information was collected after all of the MySQL tests were executed, which means the runtime information was accumulated from all tests run. There was no way to decipher how each test contributed to this accumulated runtime information. A greater degree of granularity can provide better understanding of the MySQL runtime behavior.

To collect runtime information for each MySQL test, the Skoll client still compiles MySQL as described in the previous post. Instead of running the “mysql-test-run” script with no arguments (which runs all available tests in the source tree), Skoll client calls “mysql-test-run [TESTCASE]” to run one specific test. After this test finishes, Skoll client collects the runtime information and then repeats the process for other tests. However, to make this work, two tasks must be handled:

Skoll client needs to know which tests are available in the MySQL source tree. The tests are located in the “mysql-test/t” directory. Each test is in a file “[TESTCASE].test”, and the filename is the test name (additional test suites are in “mysql-test/suite/[TESTSUITE]/t” directories). It’s pretty straight forward for Skoll client to browse through the directory to make a list of tests.
Runtime information from the previous test must be cleared before the next test starts, because gcov accumulates runtime information from numerous executions in data files with extension *.gcda. Therefore, after each test Skoll client must recursively go through the source directories and delete those files in order to collect per test runtime information.