Peer-to-Peer (P2P) Architecture:

Definition, Taxonomies, Examples, and Applicability

Abstract

In this document, we provide a survey of P2P (Peer-to-Peer) systems. The survey includes a definition and several taxonomies of P2P systems. This survey also includes a description of which types of applications can be built with P2P technologies and examples of P2P applications that are currently in use on the Internet. Finally, we discuss architectural trade-offs and provide guidelines for deciding whether or not a P2P architecture would be suitable to meet the requirements of a given application.

Status of This Memo

This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Copyright Notice

Copyright © 2009 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License.

Table of Contents

   1. Introduction ....................................................3
   2. Definition of a P2P System ......................................3
      2.1. Applying the P2P Definition to the DNS .....................5
      2.2. Applying the P2P Definition to SIP .........................5
      2.3. Applying the P2P Definition to P2PSIP ......................6
      2.4. Applying the P2P Definition to BitTorrent ..................7
   3. Functions in a P2P System .......................................7
   4. Taxonomies for P2P Systems ......................................8
   5. P2P Applications ...............................................10
      5.1. Content Distribution ......................................10
      5.2. Distributed Computing .....................................12
      5.3. Collaboration .............................................13
      5.4. Platforms .................................................14
   6. Architectural Trade-Offs and Guidance ..........................14
   7. Security Considerations ........................................16
   8. Acknowledgements ...............................................19
   9. IAB Members at the Time of This Writing ........................19
   10. Informative References ........................................19
   Appendix A.  Historical Background on Distributed Architectures ...25

1. Introduction

P2P (Peer-to-peer) systems have received a great deal of attention in the last few years. A large number of scientific publications investigate different aspects of P2P systems, several scientific conferences explicitly focus on P2P networking, and there is an Internet Research Task Force (IRTF) Research Group (RG) on P2P systems (the Peer-to-Peer RG). There are also several commercial and non-commercial applications that use P2P principles running on the Internet. Some of these P2P applications are among the most widely used applications on the Internet at present.

However, despite all the above, engineers designing systems or developing protocol specifications do not have a common understanding of P2P systems. More alarming is the fact that many people in the telecom and datacom industries believe that P2P is synonymous with illegal activity, such as the illegal exchange of content over the Internet or P2P botnets.

The goal of this document is to discuss the trade-offs involved in deciding whether a particular application can be best designed and implemented using a P2P paradigm or a different model (e.g., a client-server paradigm). The document also aims to provide architectural guidelines to assist in making such decisions. This document provides engineers with a high-level understanding of what defines a P2P system, what types of P2P systems exist, the characteristics that can be expected from such systems, and what types of applications can be implemented using P2P technologies. Such understanding is essential in order to appreciate the trade-offs referred to above. In addition, we stress the importance of the fact that P2P systems can be used to implement perfectly legitimate applications and business models by providing several examples throughout the document.

2. Definition of a P2P System

In order to discuss P2P systems, we first need a working definition of a P2P system. In this section, we provide such a definition. All discussions in this document apply to systems that comply with that definition. In addition to providing examples of P2P systems, we provide a few examples of systems that comply only partially with the definition and, thus, cannot be strictly considered P2P systems. Since these systems are not fully P2P compliant, some of the discussions in this document may apply to them while others may not. We have chosen to include those examples anyway to stress the fact that P2P and centralized architectures are not completely disjoint alternatives. There are many examples of systems that fall, for instance, somewhere in between a pure P2P system and a centralized one.

P2P is a term used in many contexts, sometimes with slightly different meanings. It is possible to find several alternative definitions, which are not all fully equivalent, in the existing scientific literature. If we include other material (e.g., marketing material) in our search for a definition on P2P, the diversity of definitions is even higher.

The issue is that there is no clear border between a P2P paradigm and other supposedly opposite paradigms such as client-server [Milojicic2002]. In the extremes, some architectures are clearly P2P while others are clearly client-server. However, there are architectures that can be considered to be either or both, depending on the definition for P2P being considered. Consequently, it is important to understand what is common to all definitions of P2P and what are the non-common traits some authors include in their own definitions.

We consider a system to be P2P if the elements that form the system share their resources in order to provide the service the system has been designed to provide. The elements in the system both provide services to other elements and request services from other elements.

In principle, all the elements in the system should meet the previous criteria for the system to be considered P2P. However, in practice, a system can have a few exceptions (i.e., a few nodes that do not meet the criteria) and still be considered P2P. For example, a P2P system can still be considered P2P even if it has a centralized enrollment server. On the other hand, some systems divide endpoints between peers and clients. Peers both request and provide services while clients generally only request services. A system where most endpoints behaved as clients could not strictly be considered P2P.

Although most definitions do not state it explicitly, many implicitly assume that for a system to be P2P, its nodes need to be involved in transactions that are related to services that do not directly benefit the nodes.

Some authors add that the elements that form the P2P system, which unsurprisingly are called peers, should be able to communicate directly between themselves without passing intermediaries [Schollmeier2001]. Other authors add that the system should be self organizing and have decentralized control [Roussopoulus2004].

Note that the previous definitions are given within the context of a single individual service. A complex service can be made up of several individual services. Some of these individual services can consist of P2P services and some of them can consist of client-server services. For example, a file sharing client may include a P2P client to perform the actual file sharing and a web browser to access additional information on a centralized web server. Additionally, there are architectures where a client-server system can serve as a fallback for a service normally provided by a P2P system, or vice versa.

Providing a service typically involves processing or storing data. According to our definition, in a P2P system, peers share their processing and storage capacity (i.e., their hardware and software resources) so that the system can provide a service. For example, if the service to be provided is a file distribution service, different peers within the system will store different files. When a given peer wants to get a particular file, the peer will first discover which peer or peers have that file and then obtain the file from those peers.

The definition for P2P provides us with a criterion to decide whether or not a system is P2P. As examples, in the following sections we apply the definition to the DNS, SIP, P2PSIP, and BitTorrent and discuss which of these systems are P2P.

2.1. Applying the P2P Definition to the DNS

The DNS is a hierarchical distributed system that has sometimes been classified as a hierarchical client-server system and sometimes as a P2P system [Milojicic2002]. According to our definition, the DNS is not a P2P system because DNS resolvers are service requesters but not service providers. The elements in a system need to be both service requesters and service providers for the system to be considered P2P.

2.2. Applying the P2P Definition to SIP

SIP [RFC3261] is a rendezvous protocol that allows a user to locate a remote user and establish a communication session with that remote user. Once the remote user is located, sessions are established in a similar way in all SIP systems: directly between the nodes involved in the session. However, the rendezvous function can be implemented in different ways: the traditional SIP way and the P2P way. This section discusses the former. Section 2.3 discusses the latter.

In traditional SIP, a central server is typically responsible for a DNS domain. User agents in the domain register with the server. This way, when a user agent wants to communicate with a remote user agent in the same domain, the user agent consults the server, which returns the contact information of the remote user agent. Session establishment occurs directly between the user agents, without the involvement of the server.

Inter-domain communications in SIP are implemented using server federations. The servers responsible for each domain form a federation in which they can communicate with each other. This way, when a user agent wants to communicate with a remote user agent in a different domain, the user agent consults its local server, which in turn consults the server responsible for the remote user agent's domain.

SIP user agents act as both clients and servers. A given user agent can act as a client in a particular transaction and as a server in a subsequent transaction. However, traditional SIP cannot be considered a P2P system because user agents only share their resources for their own benefit. That is, a given user agent is only involved in transactions related to a service that benefits (somehow) the user agent itself. For example, any given user agent is only involved in SIP INVITE transactions intended to establish sessions that involve the user agent. For a system to be P2P, its nodes need to be involved in transactions that benefit others, that is, transactions that are related to services that do not benefit the nodes directly.

2.3. Applying the P2P Definition to P2PSIP

In addition to the traditional way of using SIP, SIP can also be used in a way that is generally referred to as P2PSIP (P2PSIP is the name of the IETF working group developing the technology). In P2PSIP, user agents do not register their contact information with a central server. Instead, they register it with an overlay formed by the user agents in the system. This way, when a user agent wants to communicate with a remote user agent, the user agent consults the overlay, which returns the contact information of the remote user agent. Session establishment occurs, as usual, directly between the user agents. P2PSIP is a P2P system because nodes share their resources by storing data that is not related to them (i.e., contact information of different user agents) and are involved in transactions that are related to services that do not revert directly to the nodes themselves (e.g., the rendezvous of two remote user agents).

2.4. Applying the P2P Definition to BitTorrent

BitTorrent [BitTorrent] is a protocol used to distribute files. The group of endpoints involved in the distribution of a particular file is called a swarm. The file is divided into several pieces. An endpoint interested in the file needs to download all the pieces of the file from other endpoints in the swarm. Endpoints downloading pieces of the file also upload pieces they already have to other endpoints in the swarm. An endpoint that both downloads (because it does not have the complete file yet) and uploads pieces is called a leecher (note that this definition is counterintuitive because, in other contexts, a leecher normally means someone that takes but does not give). When an endpoint has the whole file (i.e., it has all the pieces of the file), it does not need to download any pieces any longer. Therefore, it only uploads pieces to other endpoints. Such an endpoint is called a seeder.

BitTorrent systems are P2P systems because endpoints request services from other endpoints (i.e., download pieces from other endpoints) and provide services to other endpoints (i.e., upload pieces to other endpoints). Note, however, that a particular swarm where most endpoints were infrastructure nodes that had the complete file from the beginning and, thus, acted all the time as seeders could not be strictly considered a P2P system because most endpoints would only be providing services, not requesting them.

3. Functions in a P2P System

P2P systems include several functions. The following functions are independent of the service provided by the P2P system. They handle how peers connect to the system.

• Enrollment function: nodes joining a P2P system need to obtain valid credentials to join the system. The enrollment function handles node authentication and authorization.

• Peer discovery function: in order to join a P2P system (i.e., to become a peer), a node needs to establish a connection with one or more peers that are already part of the system. The peer discovery function allows nodes to discover peers in the system in order to connect to them.

The functions above are provided in a centralized way in some P2P systems (e.g., through a central enrollment server and a central peer discovery server, which is sometimes called a bootstrap server). Taxonomies for P2P systems, which will be discussed in Section 4, do not consider these functions when classifying P2P systems. Instead, they classify P2P systems based on how the following set of functions are implemented.

The following functions depend on the service provided by the P2P system. That is, not all P2P systems implement all functions. For example, a P2P system used only for storing data may not implement the computing function. In another example, a P2P system used only for computing may not implement the data storage function. Also, some of these functions are implemented in a centralized way in some P2P systems.

• Data indexing function: it deals with indexing the data stored in the system.

• Data storage function: it deals with storing and retrieving data from the system.

• Computation function: it deals with the computing performed by the system. Such computing can be related to, among other things, data processing or real-time media processing.

• Message transport function: it deals with message exchanges between peers. Depending on how this function is implemented, peers can exchange protocol messages through a central server, directly between themselves, or through peers that provide overlay routing.

Depending on the service being provided, some of the functions above may not be needed. Section 5 discusses different types of P2P applications, which implement different services.

4. Taxonomies for P2P Systems

Taxonomies classify elements into groups so that they can be studied more easily. People studying similar elements can focus on common problem sets. Taxonomies also provide common terminology that is useful when discussing issues related to individual elements and groups of elements within a given taxonomy. In this section, we provide a few taxonomies for P2P systems in order to facilitate their study and to present such a common terminology.

Given that different authors cannot seem to agree on a single common definition for P2P, the fact that there are also many different taxonomies of P2P systems should not come as a surprise. While classifying P2P systems according to different traits is something normal, the fact that different authors use the same term to indicate different things (e.g., first and second generation P2P systems mean different things for different authors) sometimes confuses readers.

Arguably, the most useful classification of P2P systems has to do with the way data is indexed. That is, how the data indexing function is implemented. A P2P index can be centralized, local, or distributed [RFC4981]. With a centralized index, a central server keeps references to the data in all peers. With a local index, each peer only keeps references to its own data. With a distributed index, references to data reside at several nodes. Napster, early versions of Gnutella (up to version 0.4), and Distributed Hash Table (DHT)-based systems are examples of centralized, local, and distributed indexes, respectively.

Indexes can also be classified into semantic and semantic-free. A semantic index can capture relationships between documents and their metadata whereas a semantic-free index cannot [RFC4981]. While semantic indexes allow for richer searches, they sometimes (depending on their implementation) fail to find the data even if it is actually in the system.

Some authors classify P2P systems by their level of decentralization. Hybrid P2P systems need a central entity to provide their services while pure P2P systems can continue to provide their services even if any single peer is removed from the system [Schollmeier2001]. According to this definition, P2P systems with a centralized index are hybrid P2P systems while systems with local and distributed indexes are pure P2P systems.

Still, some authors classify pure P2P systems by the level of structure they show [Alima2005]. In unstructured systems, peers join the system by connecting themselves to any other existing peers. In structured systems, peers join the system by connecting themselves to well-defined peers based on their logical identifiers. The distinction between early unstructured systems (e.g., early versions of Gnutella), which used local indexes and had no structure at all, and structured systems (e.g., the DHT-based systems), which used distributed indexes and had a well-defined structure, was fairly clear. However, unstructured systems have evolved and now show a certain level of structure (e.g., some systems have special nodes with more functionality) and use distributed indexes. Therefore, the border between unstructured and structured is somewhat blurry.

Some authors refer to different generations of P2P systems. For some, the first, second, and third generations consist of P2P systems using centralized indexes, flooding-based searches (i.e., using local indexes), and DHTs (i.e., DHT-based distributed indexes), respectively [Foster2003]. Other authors consider that second generation systems can also have non-DHT-based distributed indexes [Zhang2006]. Yet for other authors, the first and second generations consist of P2P systems using unstructured (typically using flooding- based searched) and structured (e.g., DHT-based) routing, respectively [RFC4981]. Talking about generations of P2P systems in a technical context is not useful (as stated previously, it is more useful to classify systems based on how they index data) because different generations are defined in different ways depending on the author and because talking about generations gives the impression that later generations are better than earlier ones. Depending on the application to be implemented, a P2P system of an earlier generation may meet the application's requirements in a better way than a system of a later generation.

As discussed in Section 3, the previous taxonomies do not consider the enrollment and the peer discovery functions. For example, a pure P2P system would still be considered pure even if it had centralized enrollment and peer discovery servers.

5. P2P Applications

P2P applications developed so far can be classified into the following domains [Pourebrahimi2005] [Milojicic2002]: content distribution, distributed computing, collaboration, and platforms.

5.1. Content Distribution

When most people think of P2P, they think of file sharing. Moreover, they think of illegal file sharing where users exchange material (e.g., songs, movies, and software in digital format) they are not legally authorized to distribute. However, despite people's perception, P2P file sharing systems are not intrinsically illegal.

P2P file sharing applications provide one out of many means to store and distribute content on the Internet. HTTP [RFC2616] and FTP [RFC0959] servers are examples of other content distribution mechanisms. People would not claim that HTTP is an illegal mechanism just because a number of users upload material that cannot be legally distributed to an HTTP server where other users can download it. The same way, it is misleading to claim that P2P is illegal just because some users use it for illegal purposes.

P2P content distribution systems are used to implement legitimate applications and business models that take advantage of the characteristics of these P2P systems. Examples of legitimate uses of these systems include the distribution of pre-recorded TV programs

   [Rodriguez2005], Linux distributions [Rodriguez2005], game updates
   [WoW], and live TV [Peltotalo2008] [Octoshape] by parties legally
   authorized to distribute that content (e.g., the content owner).

The main advantage of P2P content distribution systems is their scalability. In general, the more popular the content handled, the more scalable the P2P system is. The peer that has the original content (i.e., the owner of a file or the source of an audio or video stream) distributes it to a fraction of the peers interested in the content, and these peers in turn distribute it to other peers also interested in the content. Note that, in general, there is no requirement for peers distributing content to be able to access it (e.g., the content may be encrypted so that peers without the decryption key are content distributors but not content consumers). Peers can distribute content to other peers in different ways. For example, they can distribute the whole content, pieces of the content (i.e., swarming), or linear combinations of pieces of content [Gkantsidis2005]. In any case, the end result is that the peer with the original content does not need to distribute the whole content to all the peers interested in it, as it would be the case when using a centralized server. Therefore, the capacity of the system is not limited by the processing capacity and the bandwidth of the peer with the original content and, thus, the quality of the whole service increases.

An important area that determines the characteristics of a P2P distribution system is its peer selection process. Interestingly, the different parties involved in the distribution have different views on how peers should be selected. Users are interested in connecting to peers that have the content they want and also have high bandwidth and processing capacity, and low latency so that transfers are faster. The Content Delivery Network (CDN) operator wants peers to connect first to the peers who have the rarest pieces of the content being distributed in order to improve the reliability of the system (in case those peers with the rare pieces of content leave the system). Network operators prefer peers to perform local transfers within their network so that their peering and transit agreements are not negatively affected (i.e., by downloading content from a remote network despite of the content being available locally). Sometimes, all these requirements can be met at the same time (e.g., a peer with a rare piece of content has high bandwidth and processing capacity and is in the local network). However, other times the system can just try and reach acceptable trade-offs when selecting peers. These issues were the subject of the IETF P2P Infrastructure (P2PI) workshop held in 2008.

Network operators also find that, depending on the dimensioning of their networks (e.g., where the bottlenecks are), the different traffic patterns generated by P2P or centralized CDNs can be more or less easily accommodated by the network [Huang2007].

An example of a sensor network based on P2P content distribution and Delay-tolerant Networking (DTL) is ZebraNet [Juang2002]. ZebraNet is a network used to track zebras in the wild. Each zebra carries a tracking collar that gathers data about the zebra (e.g., its position) at different times. Mobile stations communicate wirelessly with the collars in order to gather and consolidate data from different zebras. Since not all the zebras get close enough to a mobile station for their collars to be able to communicate with the station, the collars communicate among them exchanging the data they have gathered. In this way, a given collar provides the mobile station with data from different zebras, some of which may never get close enough to the mobile station. P2P networks are especially useful in situations where it is impossible to deploy a communication infrastructure (e.g., due to national park regulations or potential vandalism) such as in the previous example or when tracking reindeers in Lapland [SNC] (this project has focused on DTNs more than on P2P so far, but some of its main constraints are similar to the ones in ZebraNet). Note however that sensor networks such as ZebraNet cannot be strictly considered P2P because the only node issuing service requests (i.e., the only node interested in receiving data) is a central node (i.e., the mobile station).

5.2. Distributed Computing

In P2P distributed computing, each task is divided into independent subtasks that can be completed in parallel (i.e., no inter-task communication) and delivered to a peer. The peer completes the subtask using its resources and returns the result. When all the subtasks are completed, their results are combined to obtain the result of the original task.

Peers in P2P distributed computing systems are typically distributed geographically and are connected among them through wide-area networks. Conversely, in cluster computing, nodes in a cluster are typically physically close to each other (often in the same room) and have excellent communication capabilities among themselves. Consequently, computer clusters can divide tasks into subtasks that are not completely independent from one another and that cannot be completed in parallel. The excellent communication capabilities among the nodes in the cluster make it possible to synchronize the completion of such tasks. Since computers in a cluster are so tightly integrated, cluster computing techniques are not typically considered P2P networking.

The main advantage of P2P distributed computing systems is that a number of regular computers can deliver the performance of a much more powerful (and typically expensive) computer. Nevertheless, at present, P2P distributed computing can only be applied to tasks that can be divided into independent subtasks that can be completed in parallel. Tasks that do not show this characteristic are better performed by a single powerful computer.

Note that even though distributed computing, in general, can be considered P2P (which is why we have included it in this section as an example of a P2P application), most current systems whose main focus is distributed computing do not fully comply with the definition for P2P provided in Section 2. The reason is that, in those systems, service requests are typically generated only by a central node. That is, most nodes do not generate service requests (i.e., create tasks). This is why Grid computing [Foster1999] cannot be strictly considered P2P [Lua2005]. Another well-known example that cannot strictly be considered P2P either is SETI@home (Search for Extra-Terrestrial Intelligence) [Seti], where the resources of many computers are used to analyze radio telescope data. MapReduce [Dean2004], a programming model for processing large data sets, cannot strictly be considered P2P either, for the same reason. On the other hand, a number of collaboration applications implement distributed computing functions in a P2P way (see Section 5.3).

Another form of distributed computing that cannot be strictly considered P2P (despite its name) are P2P botnets [Grizzard2007]. In P2P botnets, service requests, which usually consist of generating spam or launching Distributed Denial-of-Service (DDoS) attacks, are typically generated by a central node (or a few central nodes); that is why they cannot be strictly considered P2P. An example of this type of P2P botnet that propagates using a DHT-based overlay is the Storm botnet [Kanich2008]. In addition to their distributed propagation techniques, some P2P botnets also use a distributed command and control channel, which makes it more difficult to combat them than traditional botnets using centralized channels [Cooke2005]. DHT-based overlays can also be used to support the configuration of different types of radio access networks [Oechsner2006].

5.3. Collaboration

P2P collaboration applications include communication applications such as Voice over IP (VoIP) and Instant Messaging (IM) applications. Section 2.3 included discussions on P2PSIP systems, which are an example of a standard-based P2P collaboration application. There are also proprietary P2P collaboration applications on the Internet [Skype]. Collaboration applications typically provide rendezvous, Network Address Translators (NAT) traversal, and a set of media- related functions (e.g., media mixing or media transcoding). Note that some of these functions (e.g., media transcoding) are, effectively, a form of distributed computing.

P2P rendezvous systems are especially useful in situations where there is no infrastructure. A few people with no Internet connectivity setting up an ad hoc system to exchange documents or the members of a recovery team communicating among themselves in a disaster area are examples of such situations. P2PSIP is sometimes referred to as infrastructureless SIP to distinguish it from traditional SIP, which relies on a rendezvous server infrastructure.

5.4. Platforms

P2P platforms can be used to build applications on top of them. They provide functionality the applications on top of them can use. An example of such a platform is JXTA [Gong2001]. JXTA provides peer discovery, grouping of peers, and communication between peers. The goal with these types of P2P platforms is that they become the preferred environment for application developers. They take advantage of the good scalability properties of P2P systems.

6. Architectural Trade-Offs and Guidance

In this document, we have provided a brief overview of P2P technologies. In order to dispel the notion that P2P technologies can only be used for illegal purposes, we have discussed a number of perfectly legitimate applications that have been implemented using P2P. Examples of these applications include video conferencing applications [Skype], the distribution of pre-recorded TV programs [Rodriguez2005], Linux distributions [Rodriguez2005], game updates [WoW], and live TV [Peltotalo2008] [Octoshape] by parties legally authorized to distribute that content.

When deciding whether or not to use a P2P architecture to implement a given application, it is important to consider the general characteristics of P2P systems and evaluate them against the application's requirements. It is not possible to provide any definitive rule to decide whether or not a particular application would be implemented best using P2P. Instead, we discuss a set of trade-offs to be considered when making architectural decisions and provide guidance on which types of requirements are better met by a P2P architecture (security-related aspects are discussed in Section 7). Ultimately, applications' operational requirements need to be analyzed on a case-by-case basis in order to decide the most suitable architecture.

P2P systems are a good option when there is no existing infrastructure and deploying it is difficult for some reason. Ad hoc systems are usually good candidates to use P2P architectures. Disaster areas where existing infrastructures have been destroyed or rendered unusable can also benefit from P2P systems.

One of the main features of P2P systems is their scalability. Since the system can leverage the processing and storage capacity of all the peers in the system, increases in the system's load are tackled by having the peers use more of their processing or storage capacity. Adding new peers generally increases the system's load but also increases the system's processing and storage capacity. That is, there is no typical need to update any central servers to be able to deal with more users or more load [Leibniz2007]. Adaptive P2P systems tune themselves in order to operate in the best possible mode when conditions such as number of peers or churn rate change [Mahajan2003]. In any case, at present, maintaining a running DHT requires nontrivial operational efforts [Rhea2005].

Robustness and reliability are important features in many systems. For many applications to be useful, it is essential that they are dependable [RFC4981]. While there are many techniques to make centralized servers highly available, peers in a P2P system are not generally expected to be highly available (of course, it is also possible to build a more expensive P2P system with only highly available peers). P2P systems are designed to cope with peers leaving the system ungracefully (e.g., by crashing). P2P systems use techniques such as data replication and redundant routing table entries to improve the system's reliability. This way, if a peer crashes, the data it stored is not lost and can still be found in the system.

The performance of a P2P system when compared to a server-based system depends on many factors (e.g., the dimensioning of the server- based system). One of the most important factors is the type of task to be performed. As we discussed in Section 5.2, if the task that needs to be computed can be divided into independent subtasks that can be completed in parallel, a P2P distributed computing system made up of regular computers may be able to perform better than even a super computer. If the task at hand consists of completing database queries, a well-dimensioned centralized database may be faster than a DHT.

The performance of a P2P system can be negatively affected by a lack of cooperation between the peers in the system. It is important to have incentives in place in order to minimize the number of free riders in the system. Incentive systems generally aim to take the P2P system to optimal levels of cooperation [Feldman2004].

There are trade-offs between the scalability, robustness, and performance of a particular P2P system that can be influenced through the configuration of the system. For example, a P2P database system where each peer stored all the information in the system would be robust and have a high performance (i.e., queries would be completed quickly) but would not be efficient or scalable. If the system needed to grow, it could be configured so that each node stored only a part of the information of the whole system in order to increase its efficiency and scalability at the expense of its robustness and performance.

Energy consumption is another important property of a system. Even though the overall consumption of a client-server system is generally lower than that of a P2P system providing the same service, P2P systems avoid central servers (e.g., server farms) that can potentially concentrate the consumption of high amounts of energy in a single geographical location. When the nodes in a system need to be up and running all the time anyway, it is possible to use those nodes to perform tasks in a P2P way. However, using battery-powered devices as peers in a P2P system presents some challenges because a peer typically consumes more energy than a client in a client-server architecture where they can go into sleep mode more often [Kelenyi2008]. Energy-aware P2P protocols may be the solution to these challenges [Gurun2006].

This section has discussed a set of important system properties and compared P2P and centralized systems with respect to those properties. However, the most important factor to take into consideration is often cost. Both capital and operating costs need to be taken into account when evaluating the scalability, reliability, and performance of a system. If updating a server so that it can tackle more load is inexpensive, a server-based architecture may be the best option. If a highly available server is expensive, a P2P system may be the best choice. With respect to operating costs, as previously stated, at present, maintaining a running DHT requires nontrivial operational efforts [Rhea2005].

In short, even though understanding the general properties of P2P and server-based systems is important, deciding which architecture best fits a particular application involves obtaining detailed information about the application and its context. In most scenarios, there are no easy rules that tell us when to use which architecture.

7. Security Considerations

Security is an important issue that needs to be considered when choosing an architecture to design a system. The first issue that needs to be considered is to which extent the nodes in the system can be trusted. If all the nodes in the system are fully trusted (e.g., all the nodes are under the full control of the operator of the system and will never act in a malicious or otherwise incorrect way), a P2P architecture can achieve a high level of security. However, if nodes are not fully trusted and can be expected to behave in malicious ways (e.g., launching active attacks), providing an acceptable level of security in a P2P environment becomes significantly more challenging than in a non-P2P environment because of its distributed ownership and lack of centralized control and global knowledge [Mondal2006]. Ultimately, the level of security provided by a P2P system largely depends on the proportion of its nodes that behave maliciously. Providing an acceptable level of security in a P2P system with a large number of malicious nodes can easily become impossible.

P2P systems can be used by attackers to harvest IP addresses in use. Attackers can passively obtain valid IP addresses of potential victims without performing active scans because a given peer is typically connected to multiple peers. In addition to being passive, this attack is much more efficient than performing scans when the address space to be scanned is large and sparsely populated (e.g., the current IPv6 address space). Additionally, in many cases there is a high correlation between a particular application and a particular operating system. In this way, an attacker can harvest IP addresses suitable to launch attacks that exploit vulnerabilities that are specific to a given operating system.

Central elements in centralized architectures become an obvious target for attacks. P2P systems minimize the amount of central elements and, thus, are more resilient against attacks targeted only at a few elements.

When designing a P2P system, it is important to consider a number of threats that are specific to P2P systems. Additionally, more general threats that apply to other architectures as well are sometimes bigger in a P2P environment. P2P-specific threats mainly focus on the data storage functions and the routing of P2P systems.

In a P2P system, messages (e.g., service requests) between two given peers generally traverse a set of intermediate peers that help route messages between the two peers. Those intermediate peers can attempt to launch on-path attacks they would not be able to launch if they were not on the path between the two given peers. An attacker can attempt to choose a logical location in the P2P overlay that allows it to launch on-path attacks against a particular victim or a set of victims. The Sybil [Douceur2002] attack is an example of such an attack. The attacker chooses its overlay identifier so that it allows the attacker to launch future attacks. This type of attack can be mitigated by controlling how peers obtain their identifiers (e.g., by having a central authority).

A trivial passive attack by peers routing messages consists of trying to access the contents of those messages. Encrypting message parts that are not required for routing is an obvious defense against this type of attack.

An attacker can create a message and claim that it was actually created by another peer. The attacker can even take a legitimate message as a base and modify it to launch the attack. Peer and message authentication techniques can be used to avoid this type of attack.

Attackers can attempt to launch a set of attacks against the storage function of the P2P system. The following are generic (i.e., non- P2P-specific) attacks. Even if they are generic attacks, the way to avoid or mitigate them in a P2P system can be more challenging than in other architectures.

An attacker can attempt to store too much data in the system. A quota system that can be enforced can be used to mitigate this attack.

Unauthorized peers can attempt to perform operations on data objects. Peer authorization in conjunction with peer authentication avoids unauthorized operations.

A peer can return forged data objects claiming they are legitimate. Data object authentication prevents this attack. However, a peer can return a previous version of a data object and claim it is the current version. The use of lifetimes can mitigate this type of attack.

The following are P2P-specific attacks against the data storage function of a P2P system. An attacker can refuse to store a particular data object. An attacker can also claim a particular data object does not exist even if another peer created it and stored it on the attacker. These DoS (Denial-of-Service) attacks can be mitigated by using data replication techniques and performing multiple, typically parallel, searches.

Attackers can attempt to launch a set of attacks against the routing of the P2P system. An attacker can attempt to modify the routing of the system in order to be able to launch on-path attacks. Attackers can use forged routing maintenance messages for this purpose. The Eclipse attack [Singh2006] is an example of such an attack.

Enforcing structural constraints or enforcing node degree bounds can mitigate this type of attack.

It is possible to launch DoS attacks by modifying or dropping routing maintenance messages or by creating forged ones. Having nodes get routing tables from multiple peers can help mitigate this type of attack.

Attackers can launch a DoS attack by creating churn. By leaving and joining a P2P overlay rapidly many times, a set of attackers can create large amounts of maintenance traffic and make the routing structure of the overlay unstable. Limiting the amount of churn per node is a possible defense against this attack.

8. Acknowledgements

Jouni Maenpaa and Jani Hautakorpi helped with the literature review. Henning Schulzrinne provided useful ideas on how to define P2P systems. Bruce Lowekamp, Dan Wing, Dan York, Enrico Marocco, Cullen Jennings, and Frank Uwe Andersen provided useful comments on this document. Loa Andersson, Aaron Falk, Barry Leiba, Kurtis Lindqvist, Dow Street, and Lixia Zhang participated in the IAB discussions on this document.

9. IAB Members at the Time of This Writing

Marcelo Bagnulo
Gonzalo Camarillo
Stuart Cheshire
Vijay Gill
Russ Housley
John Klensin
Olaf Kolkman
Gregory Lebovitz
Andrew Malis
Danny McPherson
David Oran
Jon Peterson
Dave Thaler

10. Informative References

   [Alima2005]         Alima, L., Ghodsi, A., and S. Haridi, "A
                       Framework for Structured Peer-to-peer Overlay
                       Networks", Global Computing, vol. 3267, Lecture
                       Notes in Computer Science: Springer Berlin /
                       Heidelberg, pp. 223-249, 2005.
   
   [BitTorrent]        Cohen, B., "The BitTorrent Protocol Specification
                       Version 11031", February 2008.
   
   [Cooke2005]         Cooke, E., Jahanian, F., and D. McPherson, "The
                       Zombie roundup: understanding, detecting, and
                       disrupting botnets", Proceedings of the Steps to
                       Reducing Unwanted Traffic on the Internet
                       Workshop, 2005.
   
   [Dean2004]          Dean, J. and S. Ghemawat, "MapReduce: Simplified
                       Data Processing on Large Clusters", Sixth
                       Symposium on Operating System Design and
                       Implementation (OSDI '04), December 2004.
   
   [Douceur2002]       Douceur, J., "The Sybil Attack", IPTPS 02,
                       March 2002.
   
   [Farber1972]        Farber, D. and K. Larson, "The Structure of a
                       Distributed Computer System - The Communications
                       System", Proceedings Symposium on Computer-
                       Communications Networks and Teletraffic,
                       Microwave Research Institute of Polytechnic
                       Institute of Brooklyn pp. 21-27, 1972.
   
   [Feldman2004]       Feldman, M., Lai, K., Stoica, I., and J. Chuang,
                       "Robust Incentive Techniques for Peer-to-peer
                       Networks", Proceedings of the 5th ACM Conference
                       on Electronic Commerce, 2004.
   
   [Foster1999]        Foster, I., "Computational Grids", Chapter 2 of
                       The Grid: Blueprint for a New
                       Computing Infrastructure, 1999.
   
   [Foster2003]        Foster, I. and A. Iamnitchi, "On Death, Taxes,
                       and the Convergence of Peer-to-Peer and Grid
                       Computing", 2nd International Workshop in Peer-
                       to-Peer Systems IPTPS '02, 2003.
   
   [Gkantsidis2005]    Gkantsidis, C. and P. Rodriguez, "Network Coding
                       for Large Scale Content Distribution", IEEE
                       INFOCOM 2005, vol. 4, March 2005.
   
   [Gong2001]          Gong, L., "JXTA: A Network Programming
                       Environment", IEEE Internet Computing, vol. 5,
                       no. 3, pp. 88-95, 2001.
   
   [Gray1983]          Gray, J. and S. Metz, "Solving the Problems of
                       Distributed Databases", Data Communications, pp.
                       183-192, 1983.
   
   [Gray1986A]         Gray, J., "An Approach to Decentralized Computer
                       Systems", IEEE Transactions on Software
                       Engineering, V 12.6, pp. 684-689, 1986.
   
   [Gray1986B]         Gray, J. and M. Anderton, "Distributed Systems:
                       Four Case Studies", IEEE Transactions on
                       Computers and Tandem Technical Report 85.5, 1986.
   
   [Grizzard2007]      Grizzard, J., Sharma, V., Nunnery, C., Kang, B.,
                       and D. Dragon, "Peer-to-peer botnets: overview
                       and case study", Proceedings of Hot Topics in
                       Understanding Botnets (HotBots '07), 2007.
   
   [Gurun2006]         Gurun, S., Nagpurkar, P., and B. Zhao, "Energy
                       Consumption and Conservation in Mobile Peer-to-
                       Peer Systems", First International Workshop on
                       Decentralized Resource Sharing in Mobile
                       Computing and Networking (MobiShare 2006), 2006.
   
   [Huang2007]         Huang, Y., Rabinovich, M., and Z. Xiao,
                       "Challenges of P2P Streaming Technologies for
                       IPTV Services", IPTC Workshop International World
                       Wide Web Conference, Edinburgh, Scotland, United
                       Kingdom, May 2006.
   
   [Juang2002]         Juang, P., Oki, H., Wang, Y., Martonosi, M., Peh,
                       L., and D. Rubenstein, "Energy-efficient
                       computing for wildlife tracking: design tradeoffs
                       and early experiences with ZebraNet", Proceedings
                       of Conference on Computer and Communications
                       Security (CCS), ACM, 2002.
   
   [Kanich2008]        Kanich, C., Levchenko, K., Enright, B., Voelker,
                       G., Paxson, V., and S. Savage, "Spamalytics: An
                       Empirical Analysis of Spam Marketing Conversion",
                       Proceedings of Conference on Computer and
                       Communications Security (CCS) (ACM),
                       October 2008.
   
   [Kelenyi2008]       Kelenyi, I. and J. Nurminen, "Energy Aspects of
                       Peer Cooperation - Measurements with a Mobile DHT
                       System", in Proc. of Cognitive and Cooperative
                       Wireless Networks Workshop in the IEEE
                       International Conference on Communications 2008,
                       Beijing, China, pp. 164-168, 2008.
   
   [Leibniz2007]       Leibniz, K., Hobfeld, T., Wakamiya, N., and M.
                       Murata, "Peer-to-Peer vs. Client/Server:
                       Reliability and Efficiency of a Content
                       Distribution Service", Lecture Notes in Computer
                       Science, LNCS 4516, pp. 1161-1172, 2007.
   
   [Lua2005]           Keong Lua, E., Crowcroft, J., Pias, M., Sharma,
                       R., and S. Lim, "A Survey and Comparison of Peer-
                       to-peer Overlay Network Schemes", IEEE
                       Communications Surveys & Tutorials, vol. 7, no.
                       2, Second Quarter 2005, pp. 72-93, 2005.
   
   [MMUSIC-ICE]        Rosenberg, J., "Interactive Connectivity
                       Establishment (ICE): A Protocol for Network
                       Address Translator (NAT) Traversal for Offer/
                       Answer Protocols", Work in Progress,
                       October 2007.
   
   [Mahajan2003]       Mahajan, R., Castro, M., and A. Rowstron,
                       "Controlling the Cost of Reliability in Peer-to-
                       Peer Overlays", Proceedings of the 2nd
                       International Workshop on Peer-to-Peer
                       Systems (IPTPS '03), 2003.
   
   [Milojicic2002]     Milojicic, D., Kalogeraki, V., Lukose, R.,
                       Nagaraja, K., Pruyne, J., Richard, B., Rollins,
                       S., and Z. Xu, "Peer-to-Peer Computing",
                       Technical Report HP, March 2002.
   
   [Mondal2006]        Mondal, A. and M. Kitsuregawa, "Privacy, Security
                       and Trust in P2P environments: A Perspective",
                       17th International Conference on Database and
                       Expert Systems Applications 2006 (DEXA '06),
                       September 2006.
   
   [Octoshape]         "Octoshape - Large Scale Live Streaming
                       Solutions", <http://www.octoshape.com>.
   
   [Oechsner2006]      Oechsner, S., Hobfeld, T., Tutschku, K.,
                       Andersen, F., and L. Caviglione, "Using Kademlia
                       for the Configuration of B3G Radio Access Nodes",
                       Proceedings of the Fourth Annual IEEE
                       International Conference on Pervasive Computing
                       and Communications Workshops (PERCOMW '06), 2006.
   
   [Peltotalo2008]     Peltotalo, J., Harju, J., Jantunen, A., Saukko,
                       M., and L. Vaatamoinen, "Peer-to-Peer Streaming
                       Technology Survey", Seventh International
                       Conference on Networking, Cancun, Mexico, pp.
                       342-350, April 2008.
   
   [Pourebrahimi2005]  Pourebrahimi, B., Bertels, K., and S.
                       Vassiliadis, "A Survey of Peer-to-Peer Networks",
                       Proceedings of the 16th Annual Workshop on
                       Circuits, Systems, and Signal Processing, ProRisc
                       2005, November 2005.
   
   [RFC0959]           Postel, J. and J. Reynolds, "File Transfer
                       Protocol", STD 9, RFC 959, October 1985.
   
   [RFC2616]           Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
                       Masinter, L., Leach, P., and T. Berners-Lee,
                       "Hypertext Transfer Protocol -- HTTP/1.1",
                       RFC 2616, June 1999.
   
   [RFC3261]           Rosenberg, J., Schulzrinne, H., Camarillo, G.,
                       Johnston, A., Peterson, J., Sparks, R., Handley,
                       M., and E. Schooler, "SIP: Session Initiation
                       Protocol", RFC 3261, June 2002.
   
   [RFC4981]           Risson, J. and T. Moors, "Survey of Research
                       towards Robust Peer-to-Peer Networks: Search
                       Methods", RFC 4981, September 2007.
   
   [RFC5128]           Srisuresh, P., Ford, B., and D. Kegel, "State of
                       Peer-to-Peer (P2P) Communication across Network
                       Address Translators (NATs)", RFC 5128,
                       March 2008.
   
   [Rhea2005]          Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J.,
                       Ratnasamy, S., Shenker, S., Stoica, I., and H.
                       Yu, "Open DHT: A Public DHT Service and Its
                       Uses", ACM/SIGCOMM CCR'05, vol. 35, Issue 4,
                       October 2005.
   
   [Rodriguez2005]     Rodriguez, P., Tan, S., and C. Gkantsidis, "On
                       the Feasibility of Commercial Legal P2P Content
                       Distribution", ACM/SIGCOMM CCR'06, January 2006.
   
   [Roussopoulus2004]  Roussopoulus, M., Baker, M., Rosenthal, D.,
                       Guili, T., Maniatis, P., and J. Mogul, "2 P2P or
                       Not 2 P2P", Workshop on Peer-to-Peer Systems,
                       February 2004.
   
   [SNC]               "http://www.snc.sapmi.net".
   
   [Schollmeier2001]   Schollmeier, R., "A Definition of Peer-to-Peer
                       Networking for the Classification of Peer-to-Peer
                       Architectures and Applications", In Proceedings
                       of the First International Conference on Peer-to-
                       Peer Computing P2P '01, 2001.
   
   [Seti]              "SETI@home", <http://setiathome.berkeley.edu>.
   
   [Singh2006]         Singh, A., Ngan, T., Druschel, T., and D.
                       Wallach, "Eclipse Attacks on Overlay Networks:
                       Threats and Defences", INFOCOM 2006, April 2006.
   
   [Skype]             "Skype", <http://www.skype.com>.
   
   [Tanenbaum1981]     Tanenbaum, A. and S. Mullender, "An Overview of
                       the Amoeba Distributed Operating System", ACM
                       SIGOPS Operating Systems Review, 1981.
   
   [WoW]               "World of Warcraft Community Site",
                       <http://www.worldofwarcraft.com>.
   
   [Zhang2006]         Zhang, Y., Chen, C., and X. Wang, "Recent
                       Advances in Research on P2P Networks", In
                       Proceedings of the Seventh International
                       Conference on Parallel and Distributed Computing,
                       Applications, and Technologies PDCAT '06, 2006.

Appendix A. Historical Background on Distributed Architectures

In this appendix, we briefly provide historical background on distributed architectures. Distributed architectures are relevant to P2P because P2P architectures are a type of distributed architecture. That is, a distributed architecture is considered P2P if it meets a set of requirements, which are discussed in Section 2.

In centralized architectures (e.g., client-server architectures), a central server (or very few central servers) undertakes most of the system's processing and storage. Conversely, decentralized architectures contain no (or very few) centralized elements.

The increasing spread of packet-switched network technologies in the 1970s made it possible to develop operational distributed computer systems [Farber1972]. Distributed computer systems received a lot of attention within the research community. Research focused on distributing the different parts of a computer system, such as its operating system [Tanenbaum1981] or its databases [Gray1983]. The idea was to hide from the user the fact that the system was distributed. That is, the user did not have to worry or even be aware of the fact that his or her files were stored in different computers or the fact that his or her tasks were processed also in a distributed way. Actions such as file transfers and task allocations were taken care of by the system in an automated fashion and were transparent to the user.

In the middle of the 1980s, building distributed computer systems using general-purpose off-the-shelf hardware and software was believed to be not much harder than building large centralized applications [Gray1986A]. It was understood that distributed systems had both advantages and disadvantages when compared to centralized systems. Choosing which type of system to use for a particular application was a trade-off that depended on the characteristics and requirements of the application [Gray1986B].

The client-server paradigm, where a client makes a request to a server that processes the request and returns the result to the client, was and is used by many Internet applications. In fact, client-server architectures were so ubiquitous on the Internet that, unfortunately, the Internet itself evolved as if the majority of the endpoints on the Internet were only interested in applications following the client-server model. With the appearance of Network Address Translators (NATs) and stateful firewalls, most Internet endpoints lost the ability to receive connections from remote endpoints unless they first initiated a connection towards those nodes. While NATs were designed not to disrupt client-server applications, distributed applications that relied on nodes receiving connections were disrupted. In a network full of NATs, these types of distributed applications could only be run among nodes with public IP addresses. Of course, most users did not like applications that only worked some of the time (i.e., when their endpoint happened to have a public IP address). Therefore, the loss of global connectivity caused by NATs was one of the reasons why applications that did not follow the client-server paradigm (e.g., P2P applications) took a relatively long time to be widely deployed on the public Internet.

The design of NAT traversal mechanisms has made it possible to deploy all types of distributed applications over a network without global connectivity. While the first NAT traversal mechanisms used by P2P applications were proprietary [RFC5128], nowadays there are standard NAT traversal mechanisms such as Interactive Connectivity Establishment (ICE) [MMUSIC-ICE]. ICE makes it possible for endpoints to establish connections among themselves in the presence of NATs. The recovery of global connectivity among Internet endpoints has made it possible to deploy many P2P applications on the public Internet (unfortunately, the fact that global connectivity is not supported natively at the network layer makes it necessary for applications to deal with NATs, which can result in highly complex systems). Some of these P2P applications have been very successful and are currently used by a large number of users.

Another factor that made it possible to deploy distributed applications was the continuous significant advances in terms of processing power and storage capacity of personal computers and networked devices. Eventually, most endpoints on the Internet had capabilities that previously were exclusively within the reach of high-end servers. The natural next step was to design distributed applications that took advantage of all that distributed available capacity.

Authors' Addresses

   Gonzalo Camarillo (editor)
   Ericsson
   Hirsalantie 11
   Jorvas  02420
   Finland
   
   EMail: Gonzalo.Camarillo@ericsson.com
   
   Internet Architecture Board

EMail:

          iab@iab.org