Sunday, August 26, 2007

Summary

Project Description

FieldStat is a tool for scanning a collection of application binaries in order to understand how a particular set of classes and methods are being used in practice.

One main purpose of the tool is to support prioritizing unit test coverage based on API usage data observed in practice. Developers can view how often and how many applications use certain types and method calls in relation to unit test coverage and difficulty of testing.
This shares similar goals with clean room software engineering in that a reliability model is based on statistical likelihood of occurring. That is, place more testing effort in more likely occurring scenarios.

The tool relies on the Mono.Cecil assembly to process CIL byte-code instructions.

Delivered

Planned
  • Get up to speed with Cecil.
  • Extract call graph.
  • Identify and isolate mono framework calls.
  • Implement CodeRank algorithm.
  • Integrate with MonoCov.
  • Test and Refine application.
  • Documentation: User manual, design docs, touch-up comments
Extra
  • Command Line and Graphics Interface
  • Export Data
  • Include other statistics such as application usage count, method complexity.
  • Plugin Architecture
  • Plugin for finding Design Fragments (Set of reoccurring system calls)
  • (External) Improve Mono.Cecil tool to have better support for reconstructing statements and message chain bad code detector.

Components

Application Repository

Understanding how API calls are used in practice requires sampling actual software. Unfortunately, access to business software is limited; however, many open source applications offer a good starting point.

Over 500 projects were downloaded from the code.google.com project. The projects were selected based on the label: Mono or CSharp. In addition, 2 projects from a company were included. The projects were manually built, or a binary distribution was acquired. Some projects had to be excluded due to immaturity(not building), misclassification, and lacking the appropriate resources to build the project.

Coverage Data

The monocov tool gathers statement coverage information from the run-time execution of a Mono application. The statement coverage can be gathered in the following manner.

In the mono distribution mcs/class/corlib directory:
> make run-test RUNTIME_FLAGS="--profile=monocov:outfile=corlib.cov,+[mscorlib]"
> monocov --export-xml=/tmp/corlib-cov corlib.cov

However, the generated xml file was intended for presentation, not importing. Instead,
a new export option is introduced.

> monocov --export-fieldstat=/tmp/corlib-cov corlib.cov

Scanners and Analysis

FieldStat uses a visitor/collector pattern for gathering statistics. A visitor class walks the assemblies, classes, and methods. To gather statistics, a collector class is registered with the visitor and is notified when a particular a node of interest is visited. For instance, a collector can be notified whenever a system call is encountered.

Some default collectors included in FieldStat.
  • AppStat - Simply counts the number of system calls used per application.
  • CodeRank - Build's an application's call tree and calculates the associated code rank of each application method.
  • TypeCount - Counts the number of times a system type and system method is statically called in an application.

Running the Tool

The simpliest way run FieldStat is to specify the directory containing the coverage information (the -export-fieldstat output from monocov).

> FieldStat --coverage-path="../../../Data/Coverage Data/coverage_output" FieldStat.exe

This would output the Results.xml files in the output/ directory.

The file can be read with a XML parser or with the DataTable.ReadFile( file ) call.

A record is as follows:

<results>
<type>System.IO.Path</type>
<method>GetFileNameWithoutExtension (string)</method>
<length>1</length>
<frequency>2</frequency>
<rankedfrequency>0.30</rankedfrequency>
<appfrequency>1</appfrequency>
<coverage>1</coverage>
</results>

Screenshots

Plugins and Future Directions

Plugin System

Plugins can be created by dropping in a file named *Plugin.dll into the Plugins/ directory.

Design Fragments

An example plugin is included in the source code as the DesignFragmentPlugin project.

A design fragment is a pattern or common usage of a framework. This plugin attempts to detect candidates for design fragments by looking at common sequences of calls. This then in turn could be used to improve the framework, or serve as documentation or snippets in how to use the framework.

For example, the following series of calls was found to occur 10 times in the application repository. It looks like the user is trying to format a date in a particular way:

System.DateTime.get_Day ()
System.Int32.ToString ()
System.DateTime.get_Month ()
System.DateTime.get_Year ()
System.Int32.ToString ()
System.String.get_Length ()
System.String.Concat (string,string)
System.String.get_Length ()
System.String.Substring (int,int)
System.String.Concat (System.String[])

The plugin is written as follows:

using FieldStat.DataCollection;
using FieldStat.CodeModel;

public class DesignFragmentPlugin : AbstractPlugin
{
public override void ComputeResults(Results results, ICollection files, Hashtable htBin)
{
Visit scan = new Visit();
scan.Collectors.Register("DesignFragment", new DesignFragmentCollector());

scan.DoScan( files, htBin);
MyCollector seqs = (MyCollector)scan.Collectors["DesignFragment"];
// Process Results ...
}
}

public class DesignFragmentCollector : AbstractCollector
{
public override void OnMethodBody(MethodBody body)
{
ArrayList seq = GetSystemCallSequences(body);
if (seq.Count > 3)
{
sequences.Add(EncodeSequenceCalls( seq ));
}
}
....
}

SystemSignature

This plugin extracts the system calls made in an application. Then the resulting system call signature is compared against the other application's signatures. The results can be clustered to find common type of applications. It may also be used to judge the implementation of a well-known type of application. For example, many irc clients have been written. How divergent is yours from the other known clients? Maybe you should have taken advantage of a different architecture that you were not aware of. (Plugin under development)

42 comments:

Wireless said...

Hello. This post is likeable, and your blog is very interesting, congratulations :-). I will add in my blogroll =). If possible gives a last there on my blog, it is about the Wireless, I hope you enjoy. The address is http://wireless-brasil.blogspot.com. A hug.

DarenPRakowski said...

免費視訊聊天室免費視訊聊天室免費視訊聊天3d美女85cc情人影城jp成人美女交友免費聊天聊天室免費視訊免費視訊聊天視訊聊天免費視訊聊天室免費聊天室免費視訊聊天室免費視訊聊天室視訊聊天室免費視訊聊天室視訊交友

美利 said...

感謝分享好的作品~~ .........................................

性份 said...

你的文章讓我有種特別的感覺,請加油哦~~........................................

明秋 said...

若對自己誠實,日積月累,就無法對別人不忠了。........................................

冠伶 said...

用心經營blog,人氣百分百~^^ 加油 ........................................

長宜芳修 said...

上床圖片av圖片免費色情圖台灣性網陰蒂淫妹貼圖色情激突成人論壇成人色情網激情成人聊天室情色影音鹹濕女生自慰影片淫慾自慰色情a片成人色情圖片網情人視訊網成人話題成人色情台灣成人淫蕩a圖片情色留言板女生奶頭情色成人全裸美女圖片情色寫真裸體照大奶子台灣色情成人網站一夜正妹美女裸體寫真g罩杯美女sex520情趣娃娃視訊打槍視訊辣妹uthome淫亂走光照av情色網女生高潮成人自拍火辣眉眉太太陰毛美女視訊聊天全裸入鏡火辣av三點全露

RussDuckett0601蕙帆 said...

笑口常開~~天天開心........................................

妍慧 said...

Failure is the mother of success...................................................

協盛 said...

要持續更新下去喲!!期待~~........................................

MathewP_Thu22269 said...

Nice post ~ 3Q..............................................................

韋于倫成 said...

人間好話,要如海綿遇水牢牢吸住..................................................

韋于倫成 said...

jp成人jp58jpkiss小說jpkissjpgirl成人jpgirljpav18japan援交少女ipkgirlintitle免費成人資訊iibabeh漫蜜室之理容院小姐的高潮h漫hot視訊honey視訊聊天honey視訊honey聊天室hk免費視訊美女hk85cchivhannelhilive影音視訊hilive會員hilive無法進入頁面hilive帳號hilive主題hilivetvhichannel免費影視hi5tv免費hi5tvhh漫hhh重鹹系列免費hhh重鹹系列金瓶梅免費網路視訊金瓶梅免費色情視訊金瓶梅免費線上聊天金瓶梅視訊聊天室辣妹視訊交友網辣妹交友視訊sex自拍sex成人uthome成人網站聊天室uthome視訊聊天室

柏懿綺辰 said...

好文,領受了!謝謝!..................................................

紋謙PhilCrispin0 said...

一個人最大的敵人常是自己。 ............................................................

佩蓉 said...

所有的資產,在不被諒解時,都成了負債..................................................

玫友 said...

你不能決定生命的長度,但你可以控制它的寬度................................................................

陳智強 said...

不要把生命看得太嚴肅,反正我們不會活著離開。..................................................................

柯凡豐 said...

人生是故事的創造與遺忘。..................................................................

李哲維 said...

Poverty tries friends..................................................................                           

江婷 said...

變天了~~注意身體,別感冒囉!.................................................................                           

堅強堅強 said...

成熟,就是有能力適應生活中的模糊。.................................................................

王名仁 said...

向著星球長驅直進的人,反比踟躕在峽路上的人,更容易達到目的。............................................................

kio said...

不只BLOG內容很棒留言也很精采 XDDDD..................................................................

懿綺懿綺 said...

rain before seven; fine before eleven.............................................................

玉苓玉苓 said...

希望我的支持可以帶給你快樂--加油.............................................................

dawsonfelicia張君dawsonfelicia均 said...

雖然不能常常來看,仍然祝你人氣百分百 ............................................................

宥妃宥妃 said...

來給你加油打氣,祝福大家開開心心。 ............................................................

王美妹 said...

真有內容的部落格~推推!............................................................

陳韋夏陳韋夏益東富益東富 said...

你的努力我們都看見了--支持你............................................................

陳晏陳詩蓁雄 said...

好的部落格,希望您能繼續堅持!!!............................................................

建張青建張青 said...

It takes all kinds to make a world.............................................................

駱李淑華明欣 said...

傻氣的人喜歡給心 雖然每次都被笑了卻得到了別人的心..................................................................

文岳仲君 said...

所有的資產,在不被諒解時,都成了負債..................................................................

吳淑惠吳淑惠吳淑惠吳淑惠 said...

一棵樹除非在春天開了花,否則難望在秋天結果。..................................................

冰微 said...

期待你的下次更新喔^____^..................................................

信陳定 said...

出遊不拘名勝,有景就是好的..................................................................

至馬馬馬馬玄馬馬馬馬 said...

婚姻對男人來說是賭他的自由,對女人而言卻是賭她的幸福。.................................................................

幸平平平平杰 said...

你不能左右天氣,但你可以改變心情.................................................................

怡靜怡靜怡靜怡雯 said...

等很久了 謝謝你的用心..................................................................

尚铭 said...

一棵樹除非在春天開了花,否則難望在秋天結果。......................................... ........................

Griya Mobil Kita said...

Nice article, thanks for the information.
sewa mobil