arena.h

This section contains reference documentation for working with protocol buffer classes in C++.

#include <google/protobuf/arena.h>
namespace google::protobuf

This file defines an Arena allocator for better allocation performance.

Classes in this file

ArenaOptions provides optional additional parameters to arena construction that control its block-allocation behavior.
Arena allocator.
Helper typetraits that indicates support for arenas in a type T at compile time.

struct ArenaOptions

#include <google/protobuf/arena.h>
namespace google::protobuf

ArenaOptions provides optional additional parameters to arena construction that control its block-allocation behavior.

Members

size_t
start_block_size
This defines the size of the first block requested from the system malloc. more...
size_t
max_block_size
This defines the maximum block size requested from system malloc (unless an individual arena allocation request occurs with a size larger than this maximum). more...
char *
initial_block
An initial block of memory for the arena to use, or NULL for none. more...
size_t
initial_block_size
The size of the initial block, if provided.
void *(*
block_alloc
A function pointer to an alloc method that returns memory blocks of size requested. more...
void(*
block_dealloc
A function pointer to a dealloc method that takes ownership of the blocks from the arena. more...
ArenaOptions()

size_t ArenaOptions::start_block_size

This defines the size of the first block requested from the system malloc.

Subsequent block sizes will increase in a geometric series up to a maximum.


size_t ArenaOptions::max_block_size

This defines the maximum block size requested from system malloc (unless an individual arena allocation request occurs with a size larger than this maximum).

Requested block sizes increase up to this value, then remain here.


char * ArenaOptions::initial_block

An initial block of memory for the arena to use, or NULL for none.

If provided, the block must live at least as long as the arena itself. The creator of the Arena retains ownership of the block after the Arena is destroyed.


void *(* ArenaOptions::block_alloc

A function pointer to an alloc method that returns memory blocks of size requested.

By default, it contains a ptr to the malloc function.

NOTE: block_alloc and dealloc functions are expected to behave like malloc and free, including Asan poisoning.


void(* ArenaOptions::block_dealloc

A function pointer to a dealloc method that takes ownership of the blocks from the arena.

By default, it contains a ptr to a wrapper function that calls free.

class Arena

#include <google/protobuf/arena.h>
namespace google::protobuf

Arena allocator.

Arena allocation replaces ordinary (heap-based) allocation with new/delete, and improves performance by aggregating allocations into larger blocks and freeing allocations all at once. Protocol messages are allocated on an arena by using Arena::CreateMessage<T>(Arena*), below, and are automatically freed when the arena is destroyed.

This is a thread-safe implementation: multiple threads may allocate from the arena concurrently. Destruction is not thread-safe and the destructing thread must synchronize with users of the arena first.

An arena provides two allocation interfaces: CreateMessage<T>, which works for arena-enabled proto2 message types as well as other types that satisfy the appropriate protocol (described below), and Create<T>, which works for any arbitrary type T. CreateMessage<T> is better when the type T supports it, because this interface (i) passes the arena pointer to the created object so that its sub-objects and internal allocations can use the arena too, and (ii) elides the object's destructor call when possible. Create<T> does not place any special requirements on the type T, and will invoke the object's destructor when the arena is destroyed.

The arena message allocation protocol, required by CreateMessage<T>(Arena* arena, Args&&... args), is as follows:

  • The type T must have (at least) two constructors: a constructor callable with args (without arena), called when a T is allocated on the heap; and a constructor callable with Arena* arena, Args&&... args, called when a T is allocated on an arena. If the second constructor is called with a NULL arena pointer, it must be equivalent to invoking the first (args-only) constructor.
  • The type T must have a particular type trait: a nested type |InternalArenaConstructable_|. This is usually a typedef to |void|. If no such type trait exists, then the instantiation CreateMessage<T> will fail to compile.
  • The type T may have the type trait |DestructorSkippable_|. If this type trait is present in the type, then its destructor will not be called if and only if it was passed a non-NULL arena pointer. If this type trait is not present on the type, then its destructor is always called when the containing arena is destroyed.

This protocol is implemented by all arena-enabled proto2 message classes as well as protobuf container types like RepeatedPtrField and Map. The protocol is internal to protobuf and is not guaranteed to be stable. Non-proto types should not rely on this protocol.

Members

const size_t
kBlockOverhead = = internal::ThreadSafeArena::kBlockHeaderSize + internal::ThreadSafeArena::kSerialArenaSize
Block overhead. more...
Arena()
Default constructor with sensible default options, tuned for average use-cases.
Arena(char * initial_block, size_t initial_block_size)
Construct an arena with default options, except for the supplied initial block. more...
explicit
Arena(const ArenaOptions & options)
Arena constructor taking custom options. more...
~Arena()
void
Init(const ArenaOptions & )
uint64
SpaceAllocated() const
The following are routines are for monitoring. more...
uint64
SpaceUsed() const
Returns the total space used by the arena. more...
uint64
Reset()
Frees all storage allocated by this arena after calling destructors registered with OwnDestructor() and freeing objects registered with Own(). more...
template void
Own(T * object)
Adds |object| to a list of heap-allocated objects to be freed with |delete| when the arena is destroyed or reset.
template void
OwnDestructor(T * object)
Adds |object| to a list of objects whose destructors will be manually called when the arena is destroyed or reset. more...
void
OwnCustomDestructor(void * object, void(*)(void *) destruct)
Adds a custom member function on an object to the list of destructors that will be manually called when the arena is destroyed or reset. more...
template static T *
CreateMessage(Arena * arena, Args &&... args)
API to create proto2 message objects on the arena. more...
template static PROTOBUF_NDEBUG_INLINE T *
Create(Arena * arena, Args &&... args)
API to create any objects on the arena. more...
template static PROTOBUF_NDEBUG_INLINE T *
CreateArray(Arena * arena, size_t num_elements)
Create an array of object type T on the arena without invoking the constructor of T. more...
template static Arena *
GetArena(const T * value)
Retrieves the arena associated with |value| if |value| is an arena-capable message, or NULL otherwise. more...

const size_t Arena::kBlockOverhead = = internal::ThreadSafeArena::kBlockHeaderSize + internal::ThreadSafeArena::kSerialArenaSize

Block overhead.

Use this as a guide for how much to over-allocate the initial block if you want an allocation of size N to fit inside it.

WARNING: if you allocate multiple objects, it is difficult to guarantee that a series of allocations will fit in the initial block, especially if Arena changes its alignment guarantees in the future!


Arena::Arena(
        char * initial_block,
        size_t initial_block_size)

Construct an arena with default options, except for the supplied initial block.

It is more efficient to use this constructor instead of passing ArenaOptions if the only configuration needed by the caller is supplying an initial block.


explicit Arena::Arena(
        const ArenaOptions & options)

Arena constructor taking custom options.

See ArenaOptions above for descriptions of the options available.


uint64 Arena::SpaceAllocated() const

The following are routines are for monitoring.

They will approximate the total sum allocated and used memory, but the exact value is an implementation deal. For instance allocated space depends on growth policies. Do not use these in unit tests. Returns the total space allocated by the arena, which is the sum of the sizes of the underlying blocks.


uint64 Arena::SpaceUsed() const

Returns the total space used by the arena.

Similar to SpaceAllocated but does not include free space and block overhead. The total space returned may not include space used by other threads executing concurrently with the call to this method.


uint64 Arena::Reset()

Frees all storage allocated by this arena after calling destructors registered with OwnDestructor() and freeing objects registered with Own().

Any objects allocated on this arena are unusable after this call. It also returns the total space used by the arena which is the sums of the sizes of the allocated blocks. This method is not thread-safe.


template void Arena::OwnDestructor(
        T * object)

Adds |object| to a list of objects whose destructors will be manually called when the arena is destroyed or reset.

This differs from Own() in that it does not free the underlying memory with |delete|; hence, it is normally only used for objects that are placement-newed into arena-allocated memory.


void Arena::OwnCustomDestructor(
        void * object,
        void(*)(void *) destruct)

Adds a custom member function on an object to the list of destructors that will be manually called when the arena is destroyed or reset.

This differs from OwnDestructor() in that any member function may be specified, not only the class destructor.


template static T * Arena::CreateMessage(
        Arena * arena,
        Args &&... args)

API to create proto2 message objects on the arena.

If the arena passed in is NULL, then a heap allocated object is returned. Type T must be a message defined in a .proto file with cc_enable_arenas set to true, otherwise a compilation error will occur.

RepeatedField and RepeatedPtrField may also be instantiated directly on an arena with this method.

This function also accepts any type T that satisfies the arena message allocation protocol, documented above.


template static PROTOBUF_NDEBUG_INLINE T *
    Arena::Create(
        Arena * arena,
        Args &&... args)

API to create any objects on the arena.

Note that only the object will be created on the arena; the underlying ptrs (in case of a proto2 message) will be still heap allocated. Proto messages should usually be allocated with CreateMessage<T>() instead.

Note that even if T satisfies the arena message construction protocol (InternalArenaConstructable_ trait and optional DestructorSkippable_ trait), as described above, this function does not follow the protocol; instead, it treats T as a black-box type, just as if it did not have these traits. Specifically, T's constructor arguments will always be only those passed to Create<T>() – no additional arena pointer is implicitly added. Furthermore, the destructor will always be called at arena destruction time (unless the destructor is trivial). Hence, from T's point of view, it is as if the object were allocated on the heap (except that the underlying memory is obtained from the arena).


template static PROTOBUF_NDEBUG_INLINE T *
    Arena::CreateArray(
        Arena * arena,
        size_t num_elements)

Create an array of object type T on the arena without invoking the constructor of T.

If arena is null, then the return value should be freed with delete[[]] x; (or ::operator delete[[]](x);). To ensure safe uses, this function checks at compile time (when compiled as C++11) that T is trivially default-constructible and trivially destructible.


template static Arena * Arena::GetArena(
        const T * value)

Retrieves the arena associated with |value| if |value| is an arena-capable message, or NULL otherwise.

If possible, the call resolves at compile time. Note that we can often devirtualize calls to value->GetArena() so usually calling this method is unnecessary.

template class Arena::InternalHelper

#include <google/protobuf/arena.h>
namespace google::protobuf

template <typename >

Members

static Arena *
GetOwningArena(const T * p)
Provides access to protected GetOwningArena to generated messages.
static Arena *
GetArenaForAllocation(const T * p)
Provides access to protected GetArenaForAllocation to generated messages.

template struct Arena::is_arena_constructable

#include <google/protobuf/arena.h>
namespace google::protobuf

template <typename >

Helper typetraits that indicates support for arenas in a type T at compile time.

This is public only to allow construction of higher-level templated utilities.

is_arena_constructable<T>::value is true if the message type T has arena support enabled, and false otherwise.

is_destructor_skippable<T>::value is true if the message type T has told the arena that it is safe to skip the destructor, and false otherwise.

This is inside Arena because only Arena has the friend relationships necessary to see the underlying generated code traits.

Members

template struct Arena::is_destructor_skippable

#include <google/protobuf/arena.h>
namespace google::protobuf

template <typename >

Members