一个偶然的机会接触到了golang,被它的高并发传说所吸引,就开始学这门语言,越学感觉越有意思^_^
注册了博客园这么多年,第一次写东西,年纪大了,脑子不好使了,就得写下来,记下来,为了自己以后查阅,同时也分享给golang的爱好者。
gnet 在CSDN上的介绍如下:https://blog.csdn.net/qq_31967569/article/details/103107707
github v1版本的开源地址为:https://github.com/panjf2000/gnet
详细介绍看这里,英文版的有各个模块的介绍,较详细:https://pkg.go.dev/github.com/panjf2000/gnet?GOOS=windows#Option
gnet作者潘少是国内一枚年轻大帅哥^_^,在知乎上对gnet这个框架有不同的观点,说什么的也有,有不同的声音,是好事,喜欢就用,不喜欢就不用,请尊重开源作者。开源作者对go社区的贡献有目共睹。下面记录一下我在使用中的一些笔记。
先记录一下Options,为便于查询,保留原始英文备注
type Options struct { // Multicore indicates whether the server will be effectively created with multi-cores, if so, // then you must take care with synchronizing memory between all event callbacks, otherwise, // it will run the server with single thread. The number of threads in the server will be automatically // assigned to the value of logical CPUs usable by the current process. // Multicore 表示服务器是否将有效地使用多核创建,如果是,则必须注意在所有事件回调之间同步内存,否则它将以单线程运行服务器。 // 服务器中的线程数将自动分配给当前进程可用的逻辑 CPU 的值。 Multicore bool // NumEventLoop is set up to start the given number of event-loop goroutine. // NumEventLoop 设置为启动给定数量的event-loop goroutine,也就是 sub reactor的数量----这个参数的说明我特意私下请教过潘少 // Note: Setting up NumEventLoop will override Multicore. // 这个要注意,如果设置了NumEventLoop的值,那么Multicored 的参数将会被覆盖,潘少给的答复是一般用NumEventLoop这个参数,Multicore和NumEventLoop用哪个参数都行,需要自行在生产压力测试的时候验证 // NumEventLoop的值设置多少合适,也是经验值,需根据实际情况调整,使用这可以放在配置文件中,便于生产环境中随时调整而不需要修改代码 NumEventLoop int // LB represents the load-balancing algorithm used when assigning new connections. LB LoadBalancing // ReuseAddr indicates whether to set up the SO_REUSEADDR socket option. ReuseAddr bool // ReusePort indicates whether to set up the SO_REUSEPORT socket option. ReusePort bool // ReadBufferCap is the maximum number of bytes that can be read from the peer when the readable event comes. // The default value is 64KB, it can be reduced to avoid starving the subsequent connections. // Note that ReadBufferCap will always be converted to the least power of two integer value greater than // or equal to its real amount. ReadBufferCap int // LockOSThread is used to determine whether each I/O event-loop is associated to an OS thread, it is useful when you // need some kind of mechanisms like thread local storage, or invoke certain C libraries (such as graphics lib: GLib) // that require thread-level manipulation via cgo, or want all I/O event-loops to actually run in parallel for a // potential higher performance. LockOSThread bool // Ticker indicates whether the ticker has been set up. Ticker bool // TCPKeepAlive sets up a duration for (SO_KEEPALIVE) socket option. TCPKeepAlive time.Duration // TCPNoDelay controls whether the operating system should delay // packet transmission in hopes of sending fewer packets (Nagle's algorithm). // The default is true (no delay), meaning that data is sent // as soon as possible after a write operation. TCPNoDelay TCPSocketOpt // SocketRecvBuffer sets the maximum socket receive buffer in bytes. SocketRecvBuffer int // SocketSendBuffer sets the maximum socket send buffer in bytes. SocketSendBuffer int // ICodec encodes and decodes TCP stream. Codec ICodec // LogPath the local path where logs will be written, this is the easiest way to set up logging, // gnet instantiates a default uber-go/zap logger with this given log path, you are also allowed to employ // you own logger during the lifetime by implementing the following log.Logger interface. // Note that this option can be overridden by the option Logger. LogPath string // LogLevel indicates the logging level, it should be used along with LogPath. LogLevel logging.Level // Logger is the customized logger for logging info, if it is not set, // then gnet will use the default logger powered by go.uber.org/zap. Logger logging.Logger }
再看一下EventHandler
type EventHandler interface { // OnInitComplete fires when the server is ready for accepting connections. // The parameter server has information and various utilities. OnInitComplete(server Server) (action Action) // OnShutdown fires when the server is being shut down, it is called right after // all event-loops and connections are closed. OnShutdown(server Server) // OnOpened fires when a new connection has been opened. // The Conn c has information about the connection such as it's local and remote address. // The parameter out is the return value which is going to be sent back to the peer. // It is usually not recommended to send large amounts of data back to the peer in OnOpened. // Note that the bytes returned by OnOpened will be sent back to the peer without being encoded. OnOpened(c Conn) (out []byte, action Action) // OnClosed fires when a connection has been closed. // The parameter err is the last known connection error. OnClosed(c Conn, err error) (action Action) // PreWrite fires just before a packet is written to the peer socket, this event function is usually where // you put some code of logging/counting/reporting or any fore operations before writing data to the peer. PreWrite(c Conn) // AfterWrite fires right after a packet is written to the peer socket, this event function is usually where // you put the []byte returned from React() back to your memory pool. AfterWrite(c Conn, b []byte) // React fires when a socket receives data from the peer. // Call c.Read() or c.ReadN(n) of Conn c to read incoming data from the peer. // The parameter out is the return value which is going to be sent back to the peer. // Note that the parameter packet returned from React() is not allowed to be passed to a new goroutine, // as this []byte will be reused within event-loop after React() returns. // If you have to use packet in a new goroutine, then you need to make a copy of buf and pass this copy // to that new goroutine. // 我们的业务处理应该就是对packet中数据的处理,需要注意的是,如果业务处理放在新的goroutine中执行,那么需要将复制packet的副本传递进去 // 具体为什么,作者没说,我猜测可能是因为切片属于引用类型,新的gorountine的生命周期会影响整个gnet的工作 React(packet []byte, c Conn) (out []byte, action Action) // Tick fires immediately after the server starts and will fire again // following the duration specified by the delay return value. Tick() (delay time.Duration, action Action) }
下面是官方给的带阻塞的使用方法,使用了pool
package main import ( "log" "time" "github.com/panjf2000/gnet" "github.com/panjf2000/gnet/pool" ) type echoServer struct { *gnet.EventServer pool *pool.WorkerPool } func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) { data := append([]byte{}, c.Read()...) c.ResetBuffer() // Use ants pool to unblock the event-loop. _ = es.pool.Submit( func () { time.Sleep(1 * time.Second) c.AsyncWrite(data) }) return } func main() { p := pool.NewWorkerPool() defer p.Release() echo := &echoServer{pool: p} log.Fatal(gnet.Serve(echo, "tcp://:9000" , gnet.WithMulticore(true))) }
在Ubuntu上压测过,单台连接过6万结点,很轻松,当然,这个连接是没有任何业务处理的。仅仅是一个echo的测试。注意,在Ubuntu上测试的时候 需要修改 ulimite的值,默认1024太小。修改ulimite值的方法自行搜索。
目前遇到的问题是,options参数为 gnet.WithMulticore(true) 时,在windows上压测的时候,从任务管理器上看,gnet创建了很多线程,即使所有客户端都断开,这些线程也不减少,这个不知道什么原因。我私下请教过作者,作者说,windows平台下仅用于开发调试,不能用于生产。