Boost共享内存/内存映射文件构建容器的封装

  1. 1.首先是对boost ipc的托管内存段进行封装,如下:
  2. 2.其次是对要构建的各个容器进行封装,这里举例map的封装:
  3. 3.在托管内存段上进行容器的构建如下:

Boost Interprocess组件提供了通过了托管内存段(managed_mapped_file & managed_shared_memory)在共享内存和内存映射文件上进行复杂数据结构构建的功能。

通过boost.ipc构造简单的POD纯数据结构, 非常的简单

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include <iostream>
#include <cstdio>
#include <boost/interprocess/managed_mapped_file.hpp>

namespace bipc = ::boost::interprocess;
typedef bipc::managed_mapped_file managed_mapped_file_t;
typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t;

struct Msg
{
uint32_t id;
char name[128];
uint32_t age;
};

int main()
{
managed_mapped_file_t obj_mapped_file(bipc::open_or_create, "./msg_map.mmap", 1024*1024);
Msg * p_msg = obj_mapped_file.find_or_construct<Msg>("msg_map")();
if(NULL == p_msg)
{
std::cerr<<"construct msg failed"<<std::endl;
return -1;
}

std::cout<<p_msg->id<<std::endl;
std::cout<<p_msg->name<<std::endl;
std::cout<<p_msg->age<<std::endl;

p_msg->id = p_msg->id + 1;
strcpy(p_msg->name, "hello world!");
p_msg->age = p_msg->age + 1;

return 0;
}

在项目过程中,更多的时候是需要在共享内存/内存映射文件上构建复杂的容器。我们知道一般

  • 容器的分配器:容器的构造是直接在进程的地址空间(堆)上进行的,容器的分配器allocator都是默认构造的,所以其分配器可认为是无状态的(stateless)。
  • boost.ipc的分配器:需要从具体的系统内核内存片段上进行内存的分配,而不是进程的内存资源,所以boost.ipc的分配器是有状态的(stateful), 言外之意,需要告知boost.ipc的分配器,在哪里进行内存分配,必须传入共享内存,或是内存映射文件。

所以要在boost.ipc上构造容器,需要将boost.ipc的分配器作为容器的allocator传入容器,让容器知道在ipc上进行内存分配和对象构造。Boost ipc的allocator内的get_segment_manager()可以返回allocator构造时传入的内存片段管理器。(注意:托管内存段(managed_mapped_file & managed_shared_memory)的内存片段管理器segment_manager 本质是一个allocator,在文章http://blog.csdn.net/anonymalias/article/details/50496563有详解)。

所以现在在boost.ipc上构建一个map,需要有两步操作(这里已mapped file举例):

  • 构建mapped_file对象。
  • 通过mapped_file对象,构建map,在构建map的时候需要传入mapped_file的分配器。

代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#include <iostream>

#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/containers/map.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/allocators/node_allocator.hpp>

#include <boost/container/string.hpp>

namespace bipc = ::boost::interprocess;
typedef bipc::managed_mapped_file managed_mapped_file_t;
typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t;

struct Msg
{
uint32_t id;
bipc::string name;
uint32_t age;
};

typedef std::pair<const uint32_t, Msg> pair_t;
typedef bipc::node_allocator<pair_t, mapped_segment_manager_t> allocator_t;
typedef std::less<uint32_t> less_t;

typedef bipc::map<uint32_t, Msg, less_t, allocator_t> msg_map_t;
typedef msg_map_t::iterator map_iter_t;

int main()
{
managed_mapped_file_t obj_mapped_file(bipc::open_or_create, "./msg_map_boost.mmap", 1024*1024);
msg_map_t *p_msg_map = obj_mapped_file.find_or_construct<msg_map_t>("msg_map")(less_t(), \
obj_mapped_file.get_segment_manager());
if(NULL == p_msg_map)
{
std::cerr<<"construct msg_map failed"<<std::endl;
return -1;
}

for(int i = 0; i < 10; ++i)
{
map_iter_t itr = p_msg_map->find(i);
if(itr == p_msg_map->end())
{
std::cout<<"not find:"<<i<<" insert:"<<i<<std::endl;
Msg msg = {i, "alias", 100+i};
p_msg_map->insert(std::pair<uint32_t, Msg>(i, msg));
}
else
{
std::cout<<"find:"<<i<<" data:"<<itr->second.name<<" "<<\
itr->second.age<<std::endl;
}
}
return 0;
}

为了代码的可读性,需要定义一堆的typedef来指定待构建map的allocator的构造传参是一个boost ipc的segment_manager。当map的value不是POD,而是嵌套其他容器是,你将会看到这样的定义:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
namespace bipc = ::boost::interprocess;
typedef bipc::managed_mapped_file managed_mapped_file_t;
typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t;

typedef bipc::node_allocator<float, mapped_segment_manager_t> vec_allocator_t;
typedef boost::container::vector<float, vec_allocator_t> vector_t;

struct Msg
{
Msg(const vec_allocator_t &vec_alloc) : score(vec_alloc){}

uint32_t id;
bipc::string name;
uint32_t age;
vector_t score;
};

typedef std::pair<const uint32_t, Msg> pair_t;
typedef bipc::node_allocator<pair_t, mapped_segment_manager_t> allocator_t;
typedef std::less<uint32_t> less_t;

typedef bipc::map<uint32_t, Msg, less_t, allocator_t> msg_map_t;
typedef msg_map_t::iterator map_iter_t;

程序员的天性(C++程序员。。。), tigerlan我的导师为了不愿意造轮子,对在boost ipc上进行容器构造的逻辑进行了封装:那我写这篇文章还有毛意义,又不是我造的轮子,其实使用过程中,还是发现一些问题,算是对自己的一个提升吧,写这篇文章的本意也是想让自己记录一下过程中遇到的问题和细节,已经和大家分享的过程。

1.首先是对boost ipc的托管内存段进行封装,如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <functional>
#include <utility>
#include <string>

typedef boost::interprocess::managed_mapped_file mapped_file_t;
typedef boost::interprocess::managed_shared_memory shared_memory_t;

template <typename segment>
class Segment
{
public:
typedef segment segment_t ;
typedef typename segment_t::segment_manager segment_manager_t ;
public:
Segment(const char * name, size_t size):
m_name(name),m_size(size),
m_segment(boost::interprocess::open_or_create, m_name.c_str(), m_size )
{ }

segment_t & operator () ()
{
return m_segment;
}
/*
...省略
*/
private:
std::string m_name;
size_t m_size;
segment_t m_segment;
};

typedef Segment< mapped_file_t > MapedFile;
typedef Segment< shared_memory_t > SharedMemory;

2.其次是对要构建的各个容器进行封装,这里举例map的封装:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
#include <boost/interprocess/allocators/node_allocator.hpp>
#include <boost/interprocess/containers/map.hpp>
#include <boost/interprocess/offset_ptr.hpp>
#include <functional>
#include <utility>

#include "segment.hpp"

namespace bip = boost::interprocess;

template <typename key_t,typename data_t, typename segment_t>
class SegmentMap
{
typedef typename segment_t::segment_manager_t segment_manager_t;
typedef typename std::pair<const key_t, data_t> pair_t;
typedef typename std::less<key_t> less_t;
typedef typename bip::node_allocator <pair_t , segment_manager_t> allocator_t;

public:
typedef typename bip::map<key_t, data_t, less_t ,allocator_t> map_t;
typedef typename map_t::iterator iterator;

public:
SegmentMap(const char * name, segment_t & segment ) :
m_segment( segment ),
m_allocator( m_segment().get_segment_manager() )
{
m_segment_map = m_segment().template find_or_construct<map_t>
(name) (less_t(),m_allocator);

assert(m_segment_map != NULL);
}

SegmentMap(segment_t & segment) :
m_segment( segment ),
m_allocator( m_segment().get_segment_manager() )
{
m_segment_map = m_segment().template find_or_construct< map_t >
( bip::anonymous_instance ) (less_t(), m_allocator);
assert(m_segment_map != NULL);
}

~SegmentMap(){};

map_t & operator () ()
{
return *m_segment_map;
}

/*
...省略
*/

private:
segment_t & m_segment;
allocator_t m_allocator;
bip::offset_ptr<map_t> m_segment_map;
};

3.在托管内存段上进行容器的构建如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#include "iostream"

#include "segment.hpp"
#include "segment_map.hpp"
#include "segment_list.hpp"

typedef MapedFile _Memory;
_Memory * g_memory = NULL;

struct Company
{
typedef SegmentList<uint32_t, _Memory> leader_list_t;
typedef SegmentList<uint32_t, _Memory> colleague_list_t;
leader_list_t leader_list;
colleague_list_t colleague_list;

Company() : leader_list(*g_memory), colleague_list(*g_memory){}
};

typedef SegmentMap<uint32_t, Company, _Memory> company_map_t;

int main()
{
g_memory = new _Memory("test.mmap", 1024 * 1024);

company_map_t * p_company_map = new company_map_t("test2", *g_memory);
company_map_t::iterator itr;

for(int i = 0; i < 10; ++i)
{
itr = (*p_company_map)().find(i);
if(itr == (*p_company_map)().end())
{
std::cout<<"not find:"<<i;

Company &obj = (*p_company_map)[i];
obj.leader_list().push_back(i + 100);

std::cout<<"insert:"<<i + 100<<std::endl;
}
else
{
std::cout<<"find:"<<i;
for(Company::leader_list_t::iterator list_itr = itr->second.leader_list().begin();
list_itr != itr->second.leader_list().end(); ++list_itr)
{
std::cout<<" value:"<<*list_itr;
}

std::cout<<std::endl;
}
}
}

可以看出利用封装过后的Segment和SegmentList, SegmentMap,进行容器的构建是很方便的一件事情,需要注意的是map的value:struct Company的构造需要传入boost ipc的分配器,以便内部的list对象进行构造。所已需要Company的默认构造函数,无参,且参数初始化列表传入全局的已初始化的托管内存段对象,否则不能map的[]操作符。