-Wclass-memaccess warning with boost::endian and gcc - c++

Question

I'm getting a -Wclass-memaccess with gcc >= 8 and I would like to know if I can safely ignore the warning.
Test case:
#include <array>
#include <boost/endian/buffers.hpp>
int main()
{
static_assert(std::is_trivial<boost::endian::big_uint16_buf_t>::value);
static_assert(std::is_trivially_copyable<boost::endian::big_uint16_buf_t>::value);
// No warning when buffer is char*
{
std::array<char, 2> buffer{0x0, 0x1};
boost::endian::big_uint16_buf_t v;
std::memcpy(&v, buffer.data(), sizeof(v));
}
// -Wclass-memaccess when buffer is std::byte*
{
std::array<std::byte, 2> buffer{std::byte{0x0}, std::byte{0x1}};
boost::endian::big_uint16_buf_t v;
std::memcpy(&v, buffer.data(), sizeof(v));
}
}
Try it on wandbox
Background: This type of code is used to decode binary network traffic and memcpy is used to be safe with strict aliasing rule.
Before using that pattern more extensively on a bigger codebase, I would like to be sure it's not flawed.
I have the impression that it's safe to copy into a boost::endian buffer. Strangely there is no warning when the buffer is a pointer to char, but only when the buffer is a pointer to std::byte
Full warning message is:
prog.cc: In function 'int main()':
prog.cc:23:49: warning: 'void* memcpy(void*, const void*, size_t)' copying an object of type 'boost::endian::big_uint16_buf_t' {aka 'class boost::endian::endian_buffer<boost::endian::order::big, short unsigned int, 16>'} with 'protected' member 'boost::endian::endian_buffer<boost::endian::order::big, short unsigned int, 16>::m_value' from an array of 'std::array<std::byte, 6>::value_type' {aka 'enum class std::byte'}; use assignment or copy-initialization instead [-Wclass-memaccess]
23 | std::memcpy(&v, buffer.data(), sizeof(v));
| ^
In file included from prog.cc:2:
/opt/wandbox/boost-1.70.0/gcc-9.1.0/include/boost/endian/buffers.hpp:375:11: note: 'boost::endian::big_uint16_buf_t' {aka 'class boost::endian::endian_buffer<boost::endian::order::big, short unsigned int, 16>'} declared here
375 | class endian_buffer< order::big, T, n_bits, align::no >
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~