( = Paper PDF,
= Presentation slides,
= Presentation video)
1.
Luisa Palechor
Characterizing (un)successful open source blockchain projects and their testing practices Masters Thesis
2022.
Abstract | BibTeX | Tags: blockchain, Smart contracts, Testing
@mastersthesis{luisa2022,
title = {Characterizing (un)successful open source blockchain projects and their testing practices},
author = {Luisa Palechor},
year = {2022},
date = {2022-09-26},
urldate = {2022-09-26},
abstract = {The most well-known blockchain applications are cryptocurrencies, e.g., Ether and Bitcoin, which both sum a market cap of more than 560 billion US dollars. Besides cryptocurrency applications, programmable blockchain allows the development of different applications, e.g., peer-to-peer selling of renewable energy from smart grids, digital rights management, and supply chain tracking and operation. These applications can be developed and deployed on the blockchain through smart contracts, which are small programs that run on the blockchain under particular conditions. As bugs in blockchain applications (in particular, cryptocurrencies) can have large financial impact, it is important to ensure that these applications are well-developed and well-tested. However, currently software development and testing practices of blockchain projects are largely unstudied. In this thesis, we study data from GitHub and CoinMarketCap to understand the characteristics of successful and unsuccessful blockchain projects and reveal the testing practices in Solidity projects with the aim of helping developers to identify projects from which they can learn, or should contribute to. In the first part of the thesis, we study data from CoinMarketCap and GitHub to gain knowledge about the characteristics of successful and unsuccessful blockchain projects. We build a random forest classifier with 320 labelled projects and metrics from 3 dimensions (activity, popularity, and complexity). We found that a large number of stars and a project’s age can help distinguish between successful and unsuccessful projects. Additionally, we found that code cloning practices tend to be common in unsuccessful projects written in Python, C++, Java and Solidity. In the second part of the thesis, we explore how quality is addressed in blockchain applications by studying how 139 open source Solidity projects are tested. We show that core development team members are the developers who usually contribute to
testing files, leaving external contributions rare. In addition, our results indicate that only functional testing is practiced among the majority of Solidity projects, with Truffle and Hardhat being the tools commonly used to test Solidity smart contracts. Moreover, security testing is a practice rarely conducted, and performance testing is not conducted at all. We finally found that audits by a third party are common in several smart contracts. Future researchers and developers can use our findings to understand what characterizes successful and unsuccessful blockchain projects and be aware of the testing practices developers conduct in open source blockchain projects.},
keywords = {blockchain, Smart contracts, Testing},
pubstate = {published},
tppubtype = {mastersthesis}
}
The most well-known blockchain applications are cryptocurrencies, e.g., Ether and Bitcoin, which both sum a market cap of more than 560 billion US dollars. Besides cryptocurrency applications, programmable blockchain allows the development of different applications, e.g., peer-to-peer selling of renewable energy from smart grids, digital rights management, and supply chain tracking and operation. These applications can be developed and deployed on the blockchain through smart contracts, which are small programs that run on the blockchain under particular conditions. As bugs in blockchain applications (in particular, cryptocurrencies) can have large financial impact, it is important to ensure that these applications are well-developed and well-tested. However, currently software development and testing practices of blockchain projects are largely unstudied. In this thesis, we study data from GitHub and CoinMarketCap to understand the characteristics of successful and unsuccessful blockchain projects and reveal the testing practices in Solidity projects with the aim of helping developers to identify projects from which they can learn, or should contribute to. In the first part of the thesis, we study data from CoinMarketCap and GitHub to gain knowledge about the characteristics of successful and unsuccessful blockchain projects. We build a random forest classifier with 320 labelled projects and metrics from 3 dimensions (activity, popularity, and complexity). We found that a large number of stars and a project’s age can help distinguish between successful and unsuccessful projects. Additionally, we found that code cloning practices tend to be common in unsuccessful projects written in Python, C++, Java and Solidity. In the second part of the thesis, we explore how quality is addressed in blockchain applications by studying how 139 open source Solidity projects are tested. We show that core development team members are the developers who usually contribute to
testing files, leaving external contributions rare. In addition, our results indicate that only functional testing is practiced among the majority of Solidity projects, with Truffle and Hardhat being the tools commonly used to test Solidity smart contracts. Moreover, security testing is a practice rarely conducted, and performance testing is not conducted at all. We finally found that audits by a third party are common in several smart contracts. Future researchers and developers can use our findings to understand what characterizes successful and unsuccessful blockchain projects and be aware of the testing practices developers conduct in open source blockchain projects.
testing files, leaving external contributions rare. In addition, our results indicate that only functional testing is practiced among the majority of Solidity projects, with Truffle and Hardhat being the tools commonly used to test Solidity smart contracts. Moreover, security testing is a practice rarely conducted, and performance testing is not conducted at all. We finally found that audits by a third party are common in several smart contracts. Future researchers and developers can use our findings to understand what characterizes successful and unsuccessful blockchain projects and be aware of the testing practices developers conduct in open source blockchain projects.
2.
Luisa Palechor; Cor-Paul Bezemer
How are Solidity smart contracts tested in open source projects? An exploratory study Inproceedings
3rd IEEE/ACM International Conference on Automation of Software Test (AST), 2022.
Abstract | BibTeX | Tags: Smart contracts, Testing
@inproceedings{PalechorAST2022,
title = {How are Solidity smart contracts tested in open source projects? An exploratory study},
author = {Luisa Palechor and Cor-Paul Bezemer},
year = {2022},
date = {2022-03-10},
urldate = {2022-03-10},
booktitle = {3rd IEEE/ACM International Conference on Automation of Software Test (AST)},
abstract = {Smart contracts are self-executing programs that are stored on the
blockchain. Once a smart contract is compiled and deployed on
the blockchain, it cannot be modified. Therefore, having a bug-
free smart contract is vital. To ensure a bug-free smart contract,
it must be tested thoroughly. However, little is known about how
developers test smart contracts in practice. Our study explores 139
open source smart contract projects that are written in Solidity
to investigate the state of smart contract testing from three di-
mensions: (1) the developers working on the tests, (2) the used
testing frameworks and testnets and (3) the type of tests that are
conducted. We found that mostly core developers of a project are
responsible for testing the contracts. Second, developers typically
use only functional testing frameworks to test a smart contract,
with Truffle being the most popular one. Finally, our results show
that functional testing is conducted in most of the studied projects
(93%), security testing is only performed in a few projects (9.4%) and
traditional performance testing is conducted in none. In addition,
we found 25 projects that mentioned or published external audit
reports.},
keywords = {Smart contracts, Testing},
pubstate = {published},
tppubtype = {inproceedings}
}
Smart contracts are self-executing programs that are stored on the
blockchain. Once a smart contract is compiled and deployed on
the blockchain, it cannot be modified. Therefore, having a bug-
free smart contract is vital. To ensure a bug-free smart contract,
it must be tested thoroughly. However, little is known about how
developers test smart contracts in practice. Our study explores 139
open source smart contract projects that are written in Solidity
to investigate the state of smart contract testing from three di-
mensions: (1) the developers working on the tests, (2) the used
testing frameworks and testnets and (3) the type of tests that are
conducted. We found that mostly core developers of a project are
responsible for testing the contracts. Second, developers typically
use only functional testing frameworks to test a smart contract,
with Truffle being the most popular one. Finally, our results show
that functional testing is conducted in most of the studied projects
(93%), security testing is only performed in a few projects (9.4%) and
traditional performance testing is conducted in none. In addition,
we found 25 projects that mentioned or published external audit
reports.
blockchain. Once a smart contract is compiled and deployed on
the blockchain, it cannot be modified. Therefore, having a bug-
free smart contract is vital. To ensure a bug-free smart contract,
it must be tested thoroughly. However, little is known about how
developers test smart contracts in practice. Our study explores 139
open source smart contract projects that are written in Solidity
to investigate the state of smart contract testing from three di-
mensions: (1) the developers working on the tests, (2) the used
testing frameworks and testnets and (3) the type of tests that are
conducted. We found that mostly core developers of a project are
responsible for testing the contracts. Second, developers typically
use only functional testing frameworks to test a smart contract,
with Truffle being the most popular one. Finally, our results show
that functional testing is conducted in most of the studied projects
(93%), security testing is only performed in a few projects (9.4%) and
traditional performance testing is conducted in none. In addition,
we found 25 projects that mentioned or published external audit
reports.