[ad_1]
This text introduces a high-performance net framework, Acquire, that may be sooner than the quickest epoll-based frameworks written in Go since it’s constructed on prime of the io_uring API and was designed from the very starting to be as quick as doable. The article covers a little bit little bit of the fundamentals needed to make use of the framework accurately and demonstrates how you can write a easy TCP software constructed on prime of it.
io_uring is a comparatively new Linux API developed to deal with I/O operations. For efficiency causes, it makes use of an asynchronous programming mannequin.
io_uring is a pair of ring buffers shared by consumer house and the kernel. The primary is used to make requests (submission queue or SQ) and the second is to report the results of their execution (completion queue or CQ). When a consumer needs to carry out an operation, resembling studying or writing to a file, it creates a submission queue entry that describes the requested operation and provides it to the tail of the submission queue.
The kernel is then notified utilizing a system name that there are requests within the queue to be processed. The kernel tries to do all of the work it has to do and places the outcome on the tail of the completion queue beneath the type of completion queue occasions.
The consumer reads them one after the other ranging from the top of the queue. To scale back the variety of system calls, you’ll be able to add a number of requests earlier than notifying the kernel.
io_uring higher-level abstraction
Whereas to make use of io_uring you should use a low-level interface it might be very painful in actual purposes, so a higher-level abstraction layer is required. There’s an official library that gives this — liburing. It’s written within the C programming language and is being actively developed on a regular basis.
When creating an software or, as in my case, a library within the Go programming language, we’ll most definitely encounter a efficiency downside associated to the excessive value of calling C language capabilities in Go utilizing the cgo mechanism.
I cared quite a bit about attaining the very best doable efficiency, so I made a decision to write down my very own abstraction layer for the lower-level io_uring interface. This required a variety of further work, after all, but it surely gave me a significantly better understanding of the mechanisms of the brand new API. I encourage anybody to try the supply code printed on GitHub.
I attempted to make the API of my io_uring abstraction similar to liburing in order that anybody aware of the official model in C might use io_uring in Go as painlessly as doable. As a part of the implementation, I used the syscall, sys/unix, and unsafe packages, which, for instance, have many mechanisms to assist working instantly with the working system or permit guide allocation of reminiscence that isn’t managed later by the rubbish collector.
Programming mannequin
To profit from the largest benefit of io_uring, which is excessive efficiency, I had to make use of a programming mannequin that doesn’t generate a big efficiency overhead whereas implementing my library.
Go builders are most likely most used to utilizing an abstraction just like the web package deal. The builders of the language have executed a superb job in designing it in order that we get a reasonably easy and clear interface that enables us to work with TCP, UDP, and Unix area sockets protocols. Sadly, transferring it one-to-one to a library utilizing io_uring would value an excessive amount of when it comes to efficiency.
The essential strategies for controlling connections are the Learn, Write, and Shut strategies of the web.Conn interface. If we analyze the supply code then we’ll discover that, for instance, each try to learn knowledge includes a system name. Nonetheless, one of many major aims of io_uring is to reduce their quantity. Subsequently, the acquire.Conn interface has Learn, Write, and Shut strategies, however their habits is totally different, and none of them is obstructing.
The Learn methodology reads knowledge that Acquire has already learn from a file descriptor and positioned it within the connection’s enter buffer. The Write methodology writes knowledge to the output buffer, which Acquire will ship asynchronously after taking management of the principle loop. The Shut methodology will mark the connection as closed and block the power to make use of the connection’s strategies, however the precise closing can even occur asynchronously.
If anybody has handled the gnet library (if not, I encourage you to familiarize your self with it), you’ll most likely shortly discover that the principle API of the Acquire library could be very related. It’s based mostly totally on the EventHandler interface and its strategies.
To higher perceive how you can construct an software utilizing the Acquire framework we’ll now write a easy software utilizing the TCP protocol. Let’s not make it something fancy — a easy echo server that can reply to the consumer with the identical knowledge packet it obtained from it within the request
So let’s write a construction that implements the EventHandler interface.
sort EventHandler struct {
server acquire.Serverlogger zerolog.Logger
overallBytesSent atomic.Uint64
}
OnStart
This methodology is known as as quickly because the server begins. We will use it to initialize and configure an software written on Acquire.
For instance, to initialize the logger or begin further duties in separate goroutines. In our implementation, we can even use it to avoid wasting a reference to the server.
func (e *EventHandler) OnStart(server acquire.Server) {
e.server = server
e.logger = zerolog.New(os.Stdout).With().Logger().Stage(zerolog.InfoLevel)
}
OnAccept
The OnAccept methodology is known as when the connection is accepted. You ought to be conscious that this methodology won’t work for UDP connections as a result of UDP is a connectionless protocol, so there isn’t a connection-accepting operation within the knowledge switch move.
Whereas this methodology is operating, the connection won’t but have knowledge within the enter buffer, so attempting to learn is pointless, however you’ll be able to already attempt to ship knowledge over the connection (helpful for server-first protocol, e.g. TIME).
It’s value remembering that neither this nor the opposite strategies must be blocked for a big period of time (with one exception) as a result of they’re referred to as in the principle loop and can block the processing of io_uring requests.
func (e *EventHandler) OnAccept(conn acquire.Conn) {
e.logger.Information().
Int("lively connections", e.server.ActiveConnections()).
Str("distant tackle", conn.RemoteAddr().String()).
Msg("New connection accepted")
}
OnRead
The tactic is known as as quickly because the bytes are learn from the file descriptor. Information could be learn by parameter strategies. Most frequently, this would be the methodology that can include probably the most enterprise logic of the applying resembling a response to a consumer request. If actions are required which will block the operation of this methodology for an prolonged interval resembling costly operational I/O then take into account configuring Acquire in order that the strategy is known as in a separate goroutine every time (utilizing a goroutine pool or unbound goroutine).
In our protocol implementation, we learn the info despatched by the consumer and ship it again.
func (e *EventHandler) OnRead(conn acquire.Conn, n int) {
e.logger.Information().
Int("bytes", n).
Str("distant tackle", conn.RemoteAddr().String()).
Msg("Bytes obtained from distant peer")var (
err error
buffer []byte
)
buffer, err = conn.Subsequent(n)
if err != nil {
return
}
_, _ = conn.Write(buffer)
}
OnWrite
Fired instantly after writing bytes to the file descriptor. Helpful, for instance, if you wish to log or depend the statistics of despatched knowledge.
For the aim of our easy software, let’s assume that after sending knowledge to the consumer, the connection to the consumer must be closed.
func (e *EventHandler) OnWrite(conn acquire.Conn, n int) {
e.overallBytesSent.Add(uint64(n))e.logger.Information().
Int("bytes", n).
Str("distant tackle", conn.RemoteAddr().String()).
Msg("Bytes despatched to distant peer")
err := conn.Shut()
if err != nil {
e.logger.Error().Err(err).Msg("Error throughout connection shut")
}
}
OnClose
Fired as quickly because the TCP connection is closed. The second parameter is beneficial to find out the explanation for closing. If the error will probably be nil, it signifies that the connection was closed by the server, and was initiated by the applying. Different values will imply the connection was terminated by the consumer or a community error.
Let’s add the situation that after dealing with sufficient visitors, our server ought to shut down.
func (e *EventHandler) OnClose(conn acquire.Conn, err error) {
log := e.logger.Information().
Str("distant tackle", conn.RemoteAddr().String())
if err != nil {
log.Err(err).Msg("Connection from distant peer closed")
} else {
log.Msg("Connection from distant peer closed by server")
}if e.overallBytesSent.Load() >= uint64(len(testData)*numberOfClients) {
e.server.AsyncShutdown()
}
}
Beginning the applying
We have already got our EventHandler implementation prepared, so it stays to run our server. This time we received’t dive into the configuration choices. To begin the Acquire server we’ll use the acquire.ListenAndServe methodology.
Its first parameter is the tackle the place we would like our server to pay attention. For the needs of our software, the localhost interface will suffice.
The suffix earlier than the tackle determines what protocol Acquire ought to use. At present, TCP and UDP protocols are supported. The following parameter is our EventHandler implementation, and the remainder of the parameters are non-obligatory configurations.
Let’s use them to set the logging stage for Acquire’s inside logger.
func major() {
runClients()err := acquire.ListenAndServe(
fmt.Sprintf("tcp://localhost:%d", port), &EventHandler{}, acquire.WithLoggerLevel(logger.WarnLevel))
if err != nil {
log.Panic(err)
}
}
Consumer-side logic
To check our server, let’s write some easy code for the purchasers of our protocol. First, let’s wait some time for the server to start out up, then connect with it and ship check knowledge. Let’s examine if the operation was profitable, and in that case, let’s attempt to learn the server response and confirm whether it is what we count on.
func runClients() {
for i := 0; i < numberOfClients; i++ {
go func() {
time.Sleep(time.Second)conn, err := web.DialTimeout("tcp", fmt.Sprintf("localhost:%d", port), time.Second)
if err != nil {
log.Panic(err)
}
n, err := conn.Write(testData)
if err != nil {
log.Panic()
}
if n != len(testData) {
log.Panic()
}
buffer := make([]byte, len(testData))
n, err = conn.Learn(buffer)
if err != nil {
log.Panic()
}
if n != len(testData) {
log.Panic()
}
}()
}
}
Lastly, let’s construct and launch our software:
{"stage":"data","lively connections":2,"distant tackle":"127.0.0.1:47074","message":"New connection accepted"}
{"stage":"data","lively connections":2,"distant tackle":"127.0.0.1:47078","message":"New connection accepted"}
{"stage":"data","bytes":4,"distant tackle":"127.0.0.1:47074","message":"Bytes obtained from distant peer"}
{"stage":"data","bytes":4,"distant tackle":"127.0.0.1:47078","message":"Bytes obtained from distant peer"}
{"stage":"data","bytes":4,"distant tackle":"127.0.0.1:47074","message":"Bytes despatched to distant peer"}
{"stage":"data","bytes":4,"distant tackle":"127.0.0.1:47078","message":"Bytes despatched to distant peer"}
{"stage":"data","distant tackle":"127.0.0.1:47074","message":"Connection from distant peer closed by server"}
{"stage":"data","distant tackle":"127.0.0.1:47078","message":"Connection from distant peer closed by server"}
{"stage":"warn","element":"shopper","employee index":0,"ring fd":8,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":1,"ring fd":9,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":2,"ring fd":10,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":3,"ring fd":11,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":4,"ring fd":12,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":5,"ring fd":13,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":6,"ring fd":14,"time":1683456909,"message":"Closing connections"}
{"stage":"warn","element":"shopper","employee index":7,"ring fd":15,"time":1683456909,"message":"Closing connections"}
Attempt it your self
I strongly encourage you to check out Acquire (and provides it a star should you prefer it). I’d additionally admire any suggestions. I wish to level out that Acquire is just not but in a steady model, so its API might bear main adjustments to satisfy as many necessities of the Go builders group as doable. Subsequently, I additionally encourage you to debate whether or not the present API is evident sufficient and whether or not it’s enough to construct any TCP or UDP purposes based mostly on Acquire.
[ad_2]
