RabbitMQ笔记(二): 并发连接数

概要

对于服务器来说,并发连接数一直是一个需要考量的问题,因此在这里做一个简单的测试。

测试

在测试前,需要准备一个客户端环境,本文的环境是:

  • CentOS 6.3
  • Python 2.7.6

NOTE: 下文中的 IP 地址 10.10.0.70 为 RabbitMQ 服务器的 IP 地址

测试: 耗尽 rabbitmq 的 socket descriptors

首先, 编写一个脚本:

exhaust_socket_descriptors.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import logging

import config

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
level=logging.INFO)
LOG = logging.getLogger(os.path.basename(__file__))


def main():
connections = []
while True:
LOG.info("Try to establish a new rabbit connection...")
connection = config.get_connection()
connection.connect()
connections.append(connection)
LOG.info("[%d] connections", len(connections))


if __name__ == '__main__':
main()

客户端执行exhaust_socket_descriptors.py, 会看到如下输出:

1
2
3
4
5
6
7
8
9
10
...前略...
2014-05-13 10:45:17,477 - exhaust_socket_descriptors.py - INFO - Try to establish a new rabbit connection...
2014-05-13 10:45:17,489 - exhaust_socket_descriptors.py - INFO - [826] connections
2014-05-13 10:45:17,489 - exhaust_socket_descriptors.py - INFO - Try to establish a new rabbit connection...
2014-05-13 10:45:17,502 - exhaust_socket_descriptors.py - INFO - [827] connections
2014-05-13 10:45:17,502 - exhaust_socket_descriptors.py - INFO - Try to establish a new rabbit connection...
2014-05-13 10:45:17,516 - exhaust_socket_descriptors.py - INFO - [828] connections
2014-05-13 10:45:17,516 - exhaust_socket_descriptors.py - INFO - Try to establish a new rabbit connection...
2014-05-13 10:45:17,528 - exhaust_socket_descriptors.py - INFO - [829] connections
2014-05-13 10:45:17,529 - exhaust_socket_descriptors.py - INFO - Try to establish a new rabbit connection...

看到似乎是连接耗尽了,在服务器端上确认一下:

1
2
3
[root@localhost vagrant]# rabbitmqctl status | grep sockets_
{sockets_limit,829},
{sockets_used,829}]},

sockets_usedsockets_limit 相同,可以确认 socket descriptors 耗尽了,因此客户端成功新连接,卡在 Try to establish a new rabbit connection... 处。但是问题在于,为什么不超时?

客户端中检查一下TCP连接个数:

1
2
[root@localhost ~]# netstat -atn | grep 10.10.0.70:5672 | wc -l
830

可以得出,socket连接数是830,大于服务器端的sockets_limit

继续,在客户端中,检查一下TCP连接状态:

1
2
[root@localhost ~]# netstat -atn | grep 10.10.0.70:5672  | awk '{print $6}' | uniq
ESTABLISHED

全部都是ESTABLISHED状态,因此服务器仅仅是阻塞了新连接,而不是拒绝新连接。这样,就产生了一个问题:

如果是使用 HAProxy 等工具搭建的集群,由于服务器依然会接受新连接,因此 HAProxy 不会认为节点已Down,最终会导致整个集群卡住

Bug

当 RabbitMQ 的 sockets_used 达到 sockets_limits 时候(连接数耗尽时),最终即使是 Consumer 也会全部阻塞,只有在 sockets_used < sockets_limit 时(释放部分连接后),才会恢复。

参见以下连接获取更多信息: http://markmail.org/message/r4yhvqc7vgfljpao

Workaround: 增加File/Socket Descriptors个数

通过官方(包括EPEL)的 deb, rpm 包安装的启动脚本,都会在rabbitmq-server 启动前 source 一次 /etc/default/rabbitmq-server 文件,因此我们可以在该文件中增加最大允许的 File/Socket Descriptors 个数。

1
echo 'ulimit -n 102400' > /etc/default/rabbitmq-server

最后,重启 RabbitMQ 服务器以生效该设置:

1
service rabbitmq-server restart

Updated

May 17, 2014

  • 更新 Bug 说明 (RabbitMQ bug26180)

RabbitMQ笔记(一): 通过Vagrant建立一个RabbitMQ服务器实验环境

安装Vagrant

官方网站上,下载并安装Vagrant

NOTE: Vagrant 1.6版本对CentOS的guest支持不好,不能正确设置网络连接,需要升级到最新版或打上下面这个补丁: Fix issue reported at mitchellh#3649

下载并导入CentOS 6.3 Box

1
vagrant box add --name centos63 'https://s3.amazonaws.com/itmat-public/centos-6.3-chef-10.14.2.box'

创建Vagrantfile

Vagrantfile 文件内容如下:

Vagrantfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "centos63"

# 一个Host-Only的Network, IP地址可以自定义, 方便访问
config.vm.network :private_network, ip: "10.10.0.70", netmask: "255.255.255.0"

# config.vm.synced_folder "../", "/vagrant"

config.vm.provider :virtualbox do |vb|
# 调节虚拟机CPU个数
vb.customize ["modifyvm", :id, "--cpus", "2"]
# 调节虚拟机内存大小
vb.customize ["modifyvm", :id, "--memory", "2048"]
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/vagrant", "1"]
end

# 自动安装脚本
config.vm.provision :shell, path: "provision.sh"
end

为了简化安装,提供一个用于自动安装的脚本provision.sh,文件内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
#!/usr/bin/env bash

# Disable yum fastestmirror plugin
sed -i.backup 's/^enabled=1/enabled=0/' /etc/yum/pluginconf.d/fastestmirror.conf
# Change yum mirror (163)
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
wget -O /etc/yum.repos.d/CentOS-Base-163.repo http://mirrors.163.com/.help/CentOS6-Base-163.repo
yum makecache
yum update

# Install EPEL repository
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm

# Install and start rabbitmq-server
yum -y install rabbitmq-server

## Automatically start
chkconfig rabbitmq-server on

## Create a sample config and enable guest user remote access
echo > /etc/rabbitmq/rabbitmq.config <<EOF
[
{rabbit, [
{loopback_users, []}
]}
].
EOF

## Enable magement plugin
/usr/lib/rabbitmq/bin/rabbitmq-plugins enable rabbitmq_management

## Start rabbitmq-server
service rabbitmq-server start

# Disable iptables
service iptables stop
chkconfig iptables off

启动虚拟机

1
vagrant up

使用SSH连接到虚拟机

使用命令(如果支持的话):

1
vagrant ssh

或使用SSH工具,例如 XShell, PuTTY 等工具,连接到 10.10.0.70:22 (IP在之前 Vagrantfile 中定义)

停止虚拟机

1
vagrant halt

From Octopress To Hexo

It’s been a long time, Octopress is slow and painful to setup, after the death of my silly hard drive, I’m too lazy so I give up setting a Octopress environment, even writing new posts.

Eventually, I’m levaing, heading for Hexo.

What’s Hexo?

Hexo is a fast, simple and powerful blog framework. It parses your posts with Markdown or other render engine and generates static files with the beautiful theme. All of these just take seconds.

Why?

  • Painless installing
  • Cleaner documentation
  • Simple configuration
  • Fast generating, deploying
  • Easy customization
  • Actively maintained

双轴倾角传感器计算坡度

令 $x$ 轴, $y$ 轴上的转角分别为 $\alpha, \beta$, 平面坡度为 $\theta$, 则:

$$ \begin{align} \theta = \arccos({ \frac {1} {\sqrt{ \tan^2{\alpha} + \tan^2{\beta} + 1 }} }) \end{align} $$

最近拿了个双轴倾角传感器玩玩,把公式和示例代码记下来备忘。

推导过程就省略了,下面是C语言描述的示例代码,其中 calc_gradient_from_pitch_roll() 函数的参数和返回值都是弧度

坡度计算示例代码
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#include <stdio.h>
#include <math.h>


#define MATH_PI (3.14159f)

#define DEGREES_TO_RADIANS(angle) ((angle) * ((MATH_PI) / 180.0f))
#define RADIANS_TO_DEGREES(radians) ((radians) * (180.0f / (MATH_PI)))


float calc_gradient_from_pitch_roll(float pitch, float roll)
{

float tan_pitch = tan(pitch);
float tan_roll = tan(roll);

return acos(1.0f / sqrt(tan_pitch * tan_pitch + tan_roll * tan_roll + 1.0f));
}

// Test
int main(int argc, const char *argv[])
{

for (;;) {
float deg_pitch, deg_roll;
printf("(in degress) pitch,roll = ");
int ret = scanf("%f,%f", &deg_pitch, &deg_roll);

if (ret == 2) {
float pitch = DEGREES_TO_RADIANS(deg_pitch);
float roll = DEGREES_TO_RADIANS(deg_roll);
float gradient = calc_gradient_from_pitch_roll(pitch, roll);

printf("(in degress) gradient = %f\n", RADIANS_TO_DEGREES(gradient));
} else if (ret == EOF) {
break;
} else {
// Print error message and clear the input buffer
printf("Illegal input\n");
setbuf(stdin, NULL);
}
}

return 0;
}