Source from: Finagle: A Protocol-Agnostic RPC System
Rendering even the simplest web page on twitter.com requires the collaboration of dozens of network services speaking many different protocols. For example, in order to render the home page, the application issues requests to the Social Graph Service, Memcached, databases, and many other network services. Each of these speaks a different protocol: Thrift, Memcached, MySQL, and so on. Additionally, many of these services speak to other services — they are both servers and clients. The Social Graph Service, for instance, provides a Thrift interface but consumes from a cluster of MySQL databases.
In such systems, a frequent cause of outages is poor interaction between components in the presence of failures; common failures include crashed hosts and extreme latency variance. These failures can cascade through the system by causing work queues to back up, TCP connections to churn, or memory and file descriptors to become exhausted. In the worst case, the user sees a Fail Whale.
CHALLENGES OF BUILDING A STABLE DISTRIBUTED SYSTEM
Sophisticated network servers and clients have many moving parts: failure detectors, load-balancers, failover strategies, and so on. These parts need to work together in a delicate balance to be resilient to the varieties of failure that occur in a large production system.
This is made especially difficult by the many different implementations of failure detectors, load-balancers, and so on, per protocol. For example, the implementation of the back-pressure strategies for Thrift differ from those for HTTP. Ensuring that heterogeneous systems converge to a stable state during an incident is extremely challenging.
OUR APPROACH
We set out to develop a single implementation of the basic components of network servers and clients that could be used for all of our protocols. Finagle is a protocol-agnostic, asynchronous Remote Procedure Call (RPC) system for the Java Virtual Machine (JVM) that makes it easy to build robust clients and servers in Java, Scala, or any JVM-hosted language. Finagle supports a wide variety of request/response- oriented RPC protocols and many classes of streaming protocols.
Finagle provides a robust implementation of:
Additionally, Finagle makes it easier to build and deploy a service that
We believe our work has paid off — we can now write and deploy a network service with much greater ease and safety.
FINAGLE AT TWITTER
Today, Finagle is deployed in production at Twitter in several front- and back-end serving systems, including our URL crawler and HTTP Proxy. We plan to continue deploying Finagle more widely.
HOW FINAGLE WORKS
Finagle is flexible and easy to use because it is designed around a few simple, composable primitives: Futures
, Services
, and Filters
.
Future
objects
In Finagle, Future
objects are the unifying abstraction for all asynchronous computation. A Future
represents a computation that may not yet have completed and that can either succeed or fail. The two most basic ways to use a Future
are to:
Composing Futures
Futures
can be combined and transformed in interesting ways, leading to the kind of compositional behavior commonly seen in functional programming languages. For instance, you can convert a Future[String]
to a Future[Int]
by using map
:
Similarly, you can use flatMap
to easily pipeline a sequence of Futures
:
In this example, User.authenticate()
is performed asynchronously;Tweet.findAllByUser()
is invoked on its eventual result. This is alternatively expressed in Scala, using the for
statement:
Handling errors and exceptions is very easy when Futures
are pipelined using flatMap
or the for
statement. In the above example, if User.authenticate()
asynchronously raises an exception, the subsequent call to Tweet.findAllByUser()
never happens. Instead, the result of the pipelined expression is still of the typeFuture[Seq[Tweet]]
, but it contains the exceptional value rather than tweets. You can respond to the exception using the onFailure
callback or other compositional techniques.
A nice property of Futures
, as compared to other asynchronous programming techniques (such as the continuation passing style), is that you an easily write clear and robust asynchronous code, even with more sophisticated operations such as scatter/gather:
Service
objects
A Service
is a function that receives a request and returns a Future
object as a response. Note that both clients and servers are represented as Service
objects.
To create a Server
, you extend the abstract Service
class and listen on a port. Here is a simple HTTP server listening on port 10000:
Building an HTTP client is even easier:
Filter
objects
Filters
are a useful way to isolate distinct phases of your application into a pipeline. For example, you may need to handle exceptions, authorization, and so forth before your Service responds to a request.
A Filter
wraps a Service
and, potentially, converts the input and output types of the Service to other types. In other words, a Filter
is a Service
transformer. Here is a filter that ensures an HTTP request has valid OAuth credentials that uses an asynchronous authenticator service:
A Filter
then decorates a Service
, as in this example:
Finagle is an open source project, available under the Apache License, Version 2.0. Source code and documentation are available on GitHub.
ACKNOWLEDGEMENTS
Finagle was originally conceived by Marius Eriksen and Nick Kallen. Other key contributors are Arya Asemanfar, David Helder, Evan Meagher, Gary McCue, Glen Sanford, Grant Monroe, Ian Ownbey, Jake Donham, James Waldrop, Jeremy Cloud, Johan Oskarsson, Justin Zhu, Raghavendra Prabhu, Robey Pointer, Ryan King, Sam Whitlock, Steve Jenson, Wanli Yang, Wilhelm Bierbaum, William Morgan, Abhi Khune, and Srini Rajagopal.
相关推荐
Finagle 是 JVM 的可扩展 RPC 系统,用于构建高并发服务器。 Finagle 为多种协议实现了统一的客户端和服务器 API,并且专为高性能和并发而设计。 Finagle 的大部分代码与协议无关,简化了新协议的实现。 Finagle 是...
dasein-util.zip,编写java应用程序的实用程序java编程的通用实用程序,通常由其他dasin项目使用。包括度量单位库、日历管理扩展、jsp工具等。
Finagle是JVM的可扩展RPC系统,用于构建高并发服务器。 Finagle为几种协议实现了统一的客户端和服务器API,并且旨在实现高性能和并发性。 Finagle的大多数代码都是协议无关的,从而简化了新协议的实现。 有关详尽...
梭子豆 围绕Finagle和Twitter Future的薄包装。 该库假定您熟悉Finagle。... finagle-clojure-template :用于使用finagle-clojure和Thrift创建新项目的lein模板。用Thrift创建一个新项目lein new fin
finagle-java-example-master-slave Finagle 示例与主从设计中的服务,管理长时间运行的任务。
Finagle HTTP 和 Thrift Clients \ Servers 示例这是如何编写 Finagle HTTP/Thrift 服务器/客户端的示例。尝试一下! 要尝试它,您应该通过以下命令运行 HTTP: $ sbt "run-main com.acme.Example.HttpServer"然后你...
Finagle ServerSet 示例 这是一个快速示例,说明如何将 finagles “新”服务器设置与 zookeeper 解析器一起使用。 通过 new 我指的是在不引用ClientBuilder或ServerBuilder api 的情况下进行设置。 这个项目是用 sbt...
使用 Finagle Redis 客户端的示例应用程序要运行示例,请在默认端口 (6379) 上本地启动 Redis,然后执行: cd finagle-redis-samplesbt run享受 !! (是) :)
finagle-redis-master-slave-sample 用于 Finagle 的简单自定义 Redis 客户端,具有手动负载平衡。 为了运行示例,需要在 :6379 上使用本地 Redis,并且代码应该能够查询正确的服务器。 当然,完整的 Redis 主/从...
Http4s Finagle
finagle-kafka, 在联网和Finagle中,Apache Kafka 客户端 finagle-kafka联网和Finagle中的Apache Kafka 客户端。概述是 Finagle Kafka 是在网联网客户端和 Twitter的 Finagle 。 它使你能够以更功能的方式处理 Kafka...
Finagle名称查找器 该项目演示了如何使用Finagle构建服务,该服务将在您输入的任何文本中标识人员,地点和组织的名称。 它主要用作教学示例,并在教授的“ Finagle Essentials”课程中使用(您可以在查看该课程的...
finagle, 一种容错协议无关RPC系统 Finagle 状态项目已经在 Twitter ( 还有很多其他组织)的生产中使用,正在积极开发和维护。 插件发布在大约每月的时间内完成了发布。 在不遵循时,将详细说明 changelogs插件,并...
Finagle HTTP 服务示例 展示如何设置 Finagle HTTP 服务的示例项目。 有关演练,请参阅 。
dropwizard-auth-ldap.zip,ldap的dropwizard身份验证模块使用jndi的ldap的dropwizard身份验证模块。
pact-jvm.zip,pact jvm服务器pact的jvm版本。支持消费者驱动的契约测试,为消费者项目提供模拟服务和DSL,并为服务提供者项目提供交互回放和验证。
curator.zip,由netflixzookeeper客户端包装和富zookeeper框架开发的馆长
finagle-metrics:将Finagle指标发送到Codahale Metrics库的简便方法
带有 Finagle 的其他多服务器 Java 示例 (6.6.2) ###示例1: 客户端 <-JSON-> HttpServer <-JSON-> HttpServer2 主服务器:端口 9001 HelperServer : 端口 9004 1)来自客户端的 JSON 请求。 这可以是 curl...
NotificationPortlet.zip,jsr-286(portlet 2.0)portlet,用于显示来自校园系统的重要通知。jsr-286portlet在uportal中实现通知。