How do memory addresses work?
In 32-bit a memory address is an hexadecimal value like 0x0F032010, right? But do those values point to bytes or to bits?
And what lies between two memory addresses like 0x0F032010 and 0x0F032011
In 32-bit a memory address is an hexadecimal value like 0x0F032010, right?
Its a number. A location in memory. It is bounded by the start and end of memory, which is starting at some value and ending at some value.
But do those values point to bytes or to bits?
It is generally accepted that addresses point to the smallest addressable unit, which is a byte. Most modern CPUs are defined this way. This is however not always the case.
And what lies between two memory addresses like 0x0F032010 and 0x0F032011
Dragons. Or nothing, as there isn't anything between them.
In C and in C++, addresses ultimately point to something that is the same size as a char -- a "byte". That is the level of addressing in the language. Whether that truly is the level of addressing in the machine at hand is a different question. The number of bits in a byte is yet another question. The standard specifies a minimal value.
A C++ (or C) address, or pointer value, is best thought of as pointing to an object, not (necessarily) to a byte. Pointer arithmetic is defined in terms of the size of the pointed-to object, so incrementing an int* value gives you a pointer to the adjacent int object (which might be, say, 4 bytes farther along in memory).
On the machine level, assuming a typical linear monolithic byte-addressed memory model, pointers are implemented as machine addresses, and each address is effectively a number that refers to a single byte. An int* pointer value contains the address of the first byte of the int object it points to. Incrementing a C++ int* pointer is implemented by adding sizeof (int) (say, 4) to the machine address.
As far as C++ is concerned, pointers are not integers; they're just pointers. You can use casts to convert between pointer values and integer values, and the result should be meaningful in terms of the underlying machine's memory model, but not much is guaranteed about the results. You can perform arithmetic on pointers, but on the language level that's not at all the same thing as integer arithmetic (though it's probably implemented as scaled integer arithmetic on the machine level).
The memory model I've described is not the only possible one, and both C and C++ are deliberately designed to allow other models. For example, you could have a model where each individual object has its own memory space, and a pointer or address is a composite value consisting of something that identifies the object plus an offset within that object. Given int x; int y;, you can compare their addresses for equality (&x == &y will be false), but the behavior of &x < &y is undefined; it's not even required that &x < &y and &y < &x have opposite values.
The C++ memory model works very nicely on top of a typical 32-bit flat memory model, and you can think of pointers as numbers. But the C++ model is sufficiently abstract that it can also work on top of other models.
You can think about pointers and addresses either in the abstract terms defined by the language, or in the concrete terms implemented by the machine. They're quite different, but ultimately compatible, mental models. Keeping them both in your head simultaneously can be tricky.
Those values are nothing in themselves, just numbers. A memory address is just a number that corresponds to a byte of memory, like the street address for your house. It's only a number.
The smallest unit an address can point to is a byte, so addresses point to bytes; you can think of the address pointing to the left side of your house; the actual house (the bits that make up the byte) is between your address (which points to the left side of your house) and your next door neighbor's address (pointing to the left side of his house). But there aren't any other addresses between there.
Related
Why do you have to set a type for pointers? Aren’t they just a placeholder for addresses and all those addresses? Therefore, won't all pointers no matter what type specified occupy an equal size of memory?
You don't have to specify a type for pointers. You can use void* everywhere, which would force you to insert an explicit type cast every single time you read something from the address pointed by the pointer, or write something to that address, or simply increment/decrement or otherwise manipulate the value of the pointer.
But people decided a long time ago that they were tired of this way of programming, and preferred typed pointers that
do not require casts
do not require always having to know the size of the pointed type (which is an issue that gets even more complicated when proper memory alignment has to be taken into consideration)
prevent you from accidentally accessing the wrong data type or advancing the pointer by the wrong number of bytes.
And yes, indeed, all data pointers, no matter what their type, occupy the same amount of memory, which is usually 4 bytes on 32-bit systems, and 8 bytes on 64-bit systems. The type of a data pointer has nothing to do with the amount of memory occupied by the pointer, and that's because no type information is stored with the pointer; the pointer type is only useful to humans and to the compiler, not to the machine.
Different types take up different amounts of memory. So when advancing a pointer (e.g. in an array), we need to take the type's size into account.
For example, because a char takes up only one byte, going to the next element means adding 0x01 to the address. But because a int takes up 4 bytes (on many architectures), getting to the next element requires adding 0x04 to the address stored in the pointer.
Now, we could have a single pointer type which simply describes an address without type information (in fact, this is what void* is for), but then every time we wanted to increment or decrement it, we'd need to give the type's size as well.
Here's some real C code which demonstrates the pains you'd go through:
#include <stdlib.h>
typedef void* pointer;
int main(void) {
pointer numbers = calloc(10, sizeof(int));
int i;
for (i = 0; i < 10; i++)
*(int*)(numbers + i * sizeof(int)) = i;
/* this could have been simply "numbers[i] = i;" */
/* ... */
return 0;
}
Three important things to notice here:
We have to multiply the index by sizeof(int) every time; adding
simply i will not do: the first iteration would correctly access the
first 4-byte integer, but the second iteration would look for the
integer which starts with the second byte of the first integer, the
third would start with the third byte of the first integer, and so
on. It's very unlikely that this is desirable!
The compiler needs to know how much information it can store in a
pointer when assigning to the address it points to. For example, if
you try to store a number greater than 2^8 in a char, the compiler
should know to truncate the number and not overwrite the next few
bytes of memory, which might extend into the next page (causing a
segmentation fault) or, worse, be used to store other data in your
program, resulting in a subtle bug.
Speaking of width, we know in our program above that numbers stores
ints -- what if we didn't? What if, for example, we tried to store an
int in the address pointed to an array of a larger data type (on some
architectures), like a long? Then our generic functions would end up
having to compare the widths of both types, probably using the
minimum of the two, and then if the type being stored is smaller than
its container you start having to worry about endianness to make sure
you align the value being stored with the correct end of the
container.
If you want evaluate pointed element value with pointer then you have to specify type of pointed element on declaration pointer. Because the compiler does not know the precise number of bytes to which the pointer refers. Machine has to compute particular bounded memory to evaluate the value.
When you learn C++, or at least when I learned it through C++ Primer, pointers were termed the "memory addresses" of the elements they point to. I'm wondering to what extent this is true.
For example, do two elements *p1 and *p2 have the property p2 = p1 + 1 or p1 = p2 + 1 if and only if they are adjacent in physical memory?
You should think of pointers as being addresses of virtual memory: modern consumer operating systems and runtime environments place at least one layer of abstraction between physical memory and what you see as a pointer value.
As for your final statement, you cannot make that assumption, even in a virtual memory address space. Pointer arithmetic is only valid within blocks of contiguous memory such as arrays. And whilst it is permissible (in both C and C++) to assign a pointer to one point past an array (or scalar), the behaviour on deferencing such a pointer is undefined. Hypothesising about adjacency in physical memory in the context of C and C++ is pointless.
Not at all.
C++ is an abstraction over the code that your computer will perform. We see this abstraction leak in a few places (class member references requiring storage, for example) but in general you will be better off if you code to the abstraction and nothing else.
Pointers are pointers. They point to things. Will they be implemented as memory addresses in reality? Maybe. They could also be optimised out, or (in the case of e.g. pointers-to-members) they could be somewhat more complex than a simple numeric address.
When you start thinking of pointers as integers that map to addresses in memory, you begin to forget for example that it's undefined to hold a pointer to an object that doesn't exist (you can't just increment and decrement a pointer willy nilly to any memory address you like).
As many answers have already mentioned, they should not be thought of as memory addresses. Check out those answers and here to get an understanding of them. Addressing your last statement
*p1 and *p2 have the property p2 = p1 + 1 or p1 = p2 + 1 if and only if they are adjacent in physical memory
is only correct if p1 and p2 are of the same type, or pointing to types of the same size.
Absolutely right to think of pointers as memory addresses. That's what they are in ALL compilers that I have worked with - for a number of different processor architectures, manufactured by a number of different compiler producers.
However, the compiler does some interesting magic, to help you along with the fact that normal memory addresses [in all modern mainstream processors at least] are byte-addresses, and the object your pointer refers to may not be exactly one byte. So if we have T* ptr;, ptr++ will do ((char*)ptr) + sizeof(T); or ptr + n is ((char*)ptr) + n*sizeof(T). This also means that your p1 == p2 + 1 requires p1 and p2 to be of the same type T, since the +1 is actually +sizeof(T)*1.
There is ONE exception to the above "pointers are memory addresses", and that is member function pointers. They are "special", and for now, please just ignore how they are actually implemented, sufficient to say that they are not "just memory addresses".
The operating system provides an abstraction of the physical machine to your program (i.e. your program runs in a virtual machine). Thus, your program does not have access to any physical resource of your computer, be it CPU time, memory, etc; it merely has to ask the OS for these resources.
In the case of memory, your program works in a virtual address space, defined by the operating system. This address space has multiple regions, such as stack, heap, code, etc. The value of your pointers represent addresses in this virtual address space. Indeed, 2 pointers to consecutive addresses will point to consecutive locations in this address space.
However, this address space is splitted by the operating system into pages and segments, which are swapped in and out from memory as required, so your pointers may or may not point to consecutive physical memory locations and is impossible to tell at runtime if that is true or not. This also depends on the policy used by the operating system for paging and segmentation.
Bottom line is that pointers are memory addresses. However, they are addresses in a virtual memory space and it is up to the operating system to decide how this is mapped to the physical memory space.
As far as your program is concerned, this is not an issue. One reason for this abstraction is to make programs believe they are the only users of the machine. Imagine the nightmare you'd have to go through if you would need to consider the memory allocated by other processes when you write your program - you don't even know which processes are going to run concurrently with yours. Also, this is a good technique to enforce security: your process cannot (well, at least shouldn't be able to) access maliciously the memory space of another process since they run in 2 different (virtual) memory spaces.
Like other variables, pointer stores a data which can be an address of memory where other data is stored.
So, pointer is a variable that have an address and may hold an address.
Note that, it is not necessary that a pointer always holds an address. It may hold a non-address ID/handle etc. Therefore, saying pointer as an address is not a wise thing.
Regarding your second question:
Pointer arithmetic is valid for contiguous chunk of memory. If p2 = p1 + 1 and both pointers are of same type then p1 and p2 points to a contiguous chunk of memory. So, the addresses p1 and p2 holds are adjacent to each other.
I think this answer has the right idea but poor terminology. What C pointers provide are the exact opposite of abstraction.
An abstraction provides a mental model that's relatively easy to understand and reason about, even if the hardware is more complex and difficult to understand or harder to reason about.
C pointers are the opposite of that. They take possible difficulties of the hardware into account even when though the real hardware is often simpler and easier to reason about. They limit your reasoning to what's allowed by a union of the most complex parts of the most complex hardware regardless of how simple the hardware at hand may actually be.
C++ pointers add one thing that C doesn't include. It allows comparing all pointers of the same type for order, even if they're not in the same array. This allows a little more of a mental model, even if it doesn't match the hardware perfectly.
Somehow answers here fail to mention one specific family of pointers - that is, pointers-to-members. Those are certainly not memory addresses.
Unless pointers are optimized out by the compiler, they are integers that store memory addresses. Their lenght depends on the machine the code is being compiled for, but they can usually be treated as ints.
In fact, you can check that out by printing the actual number stored on them with printf().
Beware, however, that type * pointer increment/decrement operations are done by the sizeof(type). See for yourself with this code (tested online on Repl.it):
#include <stdio.h>
int main() {
volatile int i1 = 1337;
volatile int i2 = 31337;
volatile double d1 = 1.337;
volatile double d2 = 31.337;
volatile int* pi = &i1;
volatile double* pd = &d1;
printf("ints: %d, %d\ndoubles: %f, %f\n", i1, i2, d1, d2);
printf("0x%X = %d\n", pi, *pi);
printf("0x%X = %d\n", pi-1, *(pi-1));
printf("Difference: %d\n",(long)(pi)-(long)(pi-1));
printf("0x%X = %f\n", pd, *pd);
printf("0x%X = %f\n", pd-1, *(pd-1));
printf("Difference: %d\n",(long)(pd)-(long)(pd-1));
}
All variables and pointers were declared volatile so as the compiler wouldn't optimize them out. Also notice that I used decrement, because the variables are placed in the function stack.
The output was:
ints: 1337, 31337
doubles: 1.337000, 31.337000
0xFAFF465C = 1337
0xFAFF4658 = 31337
Difference: 4
0xFAFF4650 = 1.337000
0xFAFF4648 = 31.337000
Difference: 8
Note that this code may not work on all compilers, specially if they do not store variables in the same order. However, what's important is that the pointer values can actually be read and printed and that decrements of one may/will decrement based on the size of the variable the pointer references.
Also note that the & and * are actual operators for reference ("get the memory address of this variable") and dereference ("get the contents of this memory address").
This may also be used for cool tricks like getting the IEEE 754 binary values for floats, by casting the float* as an int*:
#include <iostream>
int main() {
float f = -9.5;
int* p = (int*)&f;
std::cout << "Binary contents:\n";
int i = sizeof(f)*8;
while(i) {
i--;
std::cout << ((*p & (1 << i))?1:0);
}
}
Result is:
Binary contents:
11000001000110000000000000000000
Example taken from https://pt.wikipedia.org/wiki/IEEE_754. Check out on any converter.
Pointers are memory addresses, but you shouldn't assume they reflect physical address. When you see addresses like 0x00ffb500 those are logical addresses that the MMU will translate to the corresponding physical address. This is the most probable scenario, since virtual memory is the most extended memory management system, but there could be systems that manage physical address directly
The particular example you give:
For example, do two elements *p1 and *p2 have the property p2 = p1 + 1 or p1 = p2 + 1 if and only if they are adjacent in physical memory?
would fail on platforms that do not have a flat address space, such as the PIC. To access physical memory on the PIC, you need both an address and a bank number, but the latter may be derived from extrinsic information such as the particular source file. So, doing arithmetic on pointers from different banks would give unexpected results.
According to the C++14 Standard, [expr.unary.op]/3:
The result of the unary & operator is a pointer to its operand. The operand shall be an lvalue or a qualified-id. If the operand is a qualified-id naming a non-static member m of some class C with type T, the result has type “pointer to member of class C of type T” and is a prvalue designating C::m. Otherwise, if the type of the expression is T, the result has type “pointer to T” and is a prvalue that is the address of the designated object or a pointer to the designated function. [Note: In particular, the address of an object of type “cv T” is “pointer to cv T”, with the same cv-qualification. —end note ]
So this says clearly and unambiguously that pointers to object type (i.e. a T *, where T is not a function type) hold addresses.
"address" is defined by [intro.memory]/1:
The memory available to a C++ program consists of one or more sequences of contiguous bytes. Every byte has a unique address.
So an address may be anything which serves to uniquely identify a particular byte of memory.
Note: In the C++ standard terminology, memory only refers to space that is in use. It doesn't mean physical memory, or virtual memory, or anything like that. The memory is a disjoint set of allocations.
It is important to bear in mind that, although one possible way of uniquely identifying each byte in memory is to assign a unique integer to each byte of physical or virtual memory, that is not the only possible way.
To avoid writing non-portable code it is good to avoid assuming that an address is identical to an integer. The rules of arithmetic for pointers are different to the rules of arithmetic for integers anyway. Similarly, we would not say that 5.0f is the same as 1084227584 even though they have identical bit representations in memory (under IEEE754).
Also, far and near pointers ... can anyone elaborate a bit?
In C++, I have no clue on how pointers work in the direct opcode level, or on the circuit level, but I know it's memory accessing other memory, or vice-versa, etc.
But in Assembly you can use pointers as well.
Is there any notable difference here that's worth knowing, or is it the same concept? Is it applied differently on the mneumonics level of low-level microprocessor specific Assembly?
Near and far pointers were only relevant for 16 bit operating systems. Ignore them unless you really really need them. You might still find the keywords in today's compilers for backwards compatibility but they won't actually do anything. In 16-bit terms, a near pointer is a 16-bit offset where the memory segment is already known by the context and a far pointer contains both the 16-bit segment and a 16-bit offset.
In assembler a pointer simply points to a memory location. The same is true in C++, but in C++ the memory might contain an object; depending on the type of the pointer, the address may change, even though it's the same object. Consider the following:
class A
{
public:
int a;
};
class B
{
public:
int b;
};
class C : public A, B
{
public:
int c;
};
C obj;
A * pA = &obj;
B * pB = &obj;
pA and pB will not be equal! They will point to different parts of the C object; the compiler makes automatic adjustments when you cast the pointer from one type to another. Since it knows the internal layout of the class it can calculate the proper offsets and apply them.
Generally, pointer is something that allows you to access something else, because it points to it.
In a computer, the "something" and the "something else" are memory contents. Since memory is accessed by specifying its memory address, a pointer (something) is a memory location that stores the memory address of something else.
In a programming language, high level or assembler, you give memory addresses a name since a name is easier to remember than a memory address (that is usually given as a hex number). This name is the name of constant that for the compiler (high level) or the assembler (machine level) is exactly the same as the hex number, or, of a variable (a memory location) that stores the hex number.
So, there is no difference in the concept of a pointer for high level languages like C++ or low level languages like assembler.
Re near/far:
In the past, some platforms, notably 16-bit DOS and 16-bit Windows, used a concept of memory segments. Near pointers were pointer into an assumed default segment and were basically just an offset whereas far pointers contained both a segment part and an offset part and could thus represent any address in memory.
In DOS, there were a bunch of different memory models you could choose from for C/C++ in particular, one where there was just one data segment and so all data pointers were implicitly near, several where there was only one code segment, one where both code and data pointers were far, and so on.
Programming with segments is a real PITA.
In addition to what everyone said regarding near/far: the difference is that in C++, pointers are typed - the compiler knows what they are pointing at, and does some behind the scenes address arithmetics for you. For example, if you have an int *p and access p[i], the compiler adds 4*i to the value of p and accesses memory at that address. That's because an integer (in most modern OSes) is 4 bytes long. Same with pointers to structures/classes - the compiler will quietly calculate the offset of the data item within the structure and adjust the memory address accordingly.
With assembly memory access, no such luck. On assembly level, technically speaking, there's almost no notion of variable datatype. Specifically, there's practically no difference between integers and pointers. When you work with arrays of something bigger than a byte, keeping track of array item length is your responsibility.
Can anyone explain to me these pointers with a suitable example ... and when these pointers are used?
The primary example is the Intel X86 architecture.
The Intel 8086 was, internally, a 16-bit processor: all of its registers were 16 bits wide. However, the address bus was 20 bits wide (1 MiB). This meant that you couldn't hold an entire address in a register, limiting you to the first 64 kiB.
Intel's solution was to create 16-bit "segment registers" whose contents would be shifted left four bits and added to the address. For example:
DS ("Data Segment") register: 1234 h
DX ("D eXtended") register: + 5678h
------
Actual address read: 179B8h
This created the concept of 64 kiB segment. Thus a "near" pointer would just be the contents of the DX register (5678h), and would be invalid unless the DS register was already set correctly, while a "far" pointer was 32 bits (12345678h, DS followed by DX) and would always work (but was slower since you had to load two registers and then restore the DS register when done).
(As supercat notes below, an offset to DX that overflowed would "roll over" before being added to DS to get the final address. This allowed 16-bit offsets to access any address in the 64 kiB segment, not just the part that was ± 32 kiB from where DX pointed, as is done in other architectures with 16-bit relative offset addressing in some instructions.)
However, note that you could have two "far" pointers that are different values but point to the same address. For example, the far pointer 100079B8h points to the same place as 12345678h. Thus, pointer comparison on far pointers was an invalid operation: the pointers could differ, but still point to the same place.
This was where I decided that Macs (with Motorola 68000 processors at the time) weren't so bad after all, so I missed out on huge pointers. IIRC, they were just far pointers that guaranteed that all the overlapping bits in the segment registers were 0's, as in the second example.
Motorola didn't have this problem with their 6800 series of processors, since they were limited to 64 kiB, When they created the 68000 architecture, they went straight to 32 bit registers, and thus never had need for near, far, or huge pointers. (Instead, their problem was that only the bottom 24 bits of the address actually mattered, so some programmers (notoriously Apple) would use the high 8 bits as "pointer flags", causing problems when address buses expanded to 32 bits (4 GiB).)
Linus Torvalds just held out until the 80386, which offered a "protected mode" where the addresses were 32 bits, and the segment registers were the high half of the address, and no addition was needed, and wrote Linux from the outset to use protected mode only, no weird segment stuff, and that's why you don't have near and far pointer support in Linux (and why no company designing a new architecture will ever go back to them if they want Linux support). And they ate Robin's minstrels, and there was much rejoicing. (Yay...)
Difference between far and huge pointers:
As we know by default the pointers are near for example: int *p is a near pointer. Size of near pointer is 2 bytes in case of 16 bit compiler. And we already know very well size varies compiler to compiler; they only store the offset of the address the pointer it is referencing. An address consisting of only an offset has a range of 0 - 64K bytes.
Far and huge pointers:
Far and huge pointers have a size of 4 bytes. They store both the segment and the offset of the address the pointer is referencing. Then what is the difference between them?
Limitation of far pointer:
We cannot change or modify the segment address of given far address by applying any arithmetic operation on it. That is by using arithmetic operator we cannot jump from one segment to other segment.
If you will increment the far address beyond the maximum value of its offset address instead of incrementing segment address it will repeat its offset address in cyclic order. This is also called wrapping, i.e. if offset is 0xffff and we add 1 then it is 0x0000 and similarly if we decrease 0x0000 by 1 then it is 0xffff and remember there is no change in the segment.
Now I am going to compare huge and far pointers :
1.When a far pointer is incremented or decremented ONLY the offset of the pointer is actually incremented or decremented but in case of huge pointer both segment and offset value will change.
Consider the following Example, taken from HERE :
int main()
{
char far* f=(char far*)0x0000ffff;
printf("%Fp",f+0x1);
return 0;
}
then the output is:
0000:0000
There is no change in segment value.
And in case of huge Pointers :
int main()
{
char huge* h=(char huge*)0x0000000f;
printf("%Fp",h+0x1);
return 0;
}
The Output is:
0001:0000
This is because of increment operation not only offset value but segment value also change.That means segment will not change in case of far pointers but in case of huge pointer, it can move from one segment to another .
2.When relational operators are used on far pointers only the offsets are compared.In other words relational operators will only work on far pointers if the segment values of the pointers being compared are the same. And in case of huge this will not happen, actually comparison of absolute addresses takes place.Let us understand with the help of an example of far pointer :
int main()
{
char far * p=(char far*)0x12340001;
char far* p1=(char far*)0x12300041;
if(p==p1)
printf("same");
else
printf("different");
return 0;
}
Output:
different
In huge pointer :
int main()
{
char huge * p=(char huge*)0x12340001;
char huge* p1=(char huge*)0x12300041;
if(p==p1)
printf("same");
else
printf("different");
return 0;
}
Output:
same
Explanation: As we see the absolute address for both p and p1 is 12341 (1234*10+1 or 1230*10+41) but they are not considered equal in 1st case because in case of far pointers only offsets are compared i.e. it will check whether 0001==0041. Which is false.
And in case of huge pointers, the comparison operation is performed on absolute addresses that are equal.
A far pointer is never normalized but a huge pointer is normalized . A normalized pointer is one that has as much of the address as possible in the segment, meaning that the offset is never larger than 15.
suppose if we have 0x1234:1234 then the normalized form of it is 0x1357:0004(absolute address is 13574).
A huge pointer is normalized only when some arithmetic operation is performed on it, and not normalized during assignment.
int main()
{
char huge* h=(char huge*)0x12341234;
char huge* h1=(char huge*)0x12341234;
printf("h=%Fp\nh1=%Fp",h,h1+0x1);
return 0;
}
Output:
h=1234:1234
h1=1357:0005
Explanation:huge pointer is not normalized in case of assignment.But if an arithmetic operation is performed on it, it will be normalized.So, h is 1234:1234 and h1 is 1357:0005which is normalized.
4.The offset of huge pointer is less than 16 because of normalization and not so in case of far pointers.
lets take an example to understand what I want to say :
int main()
{
char far* f=(char far*)0x0000000f;
printf("%Fp",f+0x1);
return 0;
}
Output:
0000:0010
In case of huge pointer :
int main()
{
char huge* h=(char huge*)0x0000000f;
printf("%Fp",h+0x1);
return 0;
}
Output:
0001:0000
Explanation:as we increment far pointer by 1 it will be 0000:0010.And as we increment huge pointer by 1 then it will be 0001:0000 because it's offset cant be greater than 15 in other words it will be normalized.
In the old days, according to the Turbo C manual, a near pointer was merely 16 bits when your entire code and data fit in the one segment. A far pointer was composed of a segment as well as an offset but no normalisation was performed. And a huge pointer was automatically normalised. Two far pointers could conceivably point to the same location in memory but be different whereas the normalised huge pointers pointing to the same memory location would always be equal.
All of the stuff in this answer is relevant only to the old 8086 and 80286 segmented memory model.
near: a 16 bit pointer that can address any byte in a 64k segment
far: a 32 bit pointer that contains a segment and an offset. Note that because segments can overlap, two different far pointers can point to the same address.
huge: a 32 bit pointer in which the segment is "normalised" so that no two far pointers point to the same address unless they have the same value.
tee: a drink with jam and bread.
That will bring us back to doh oh oh oh
and when these pointers are used?
in the 1980's and 90' until 32 bit Windows became ubiquitous,
In some architectures, a pointer which can point to every object in the system will be larger and slower to work with than one which can point to a useful subset of things. Many people have given answers related to the 16-bit x86 architecture. Various types of pointers were common on 16-bit systems, though near/fear distinctions could reappear in 64-bit systems, depending upon how they're implemented (I wouldn't be surprised if many development systems go to 64-bit pointers for everything, despite the fact that in many cases that will be very wasteful).
In many programs, it's pretty easy to subdivide memory usage into two categories: small things which together total up to a fairly small amount of stuff (64K or 4GB) but will be accessed often, and larger things which may total up to much larger quantity, but which need not be accessed so often. When an application needs to work with part of an object in the "large things" area, it copies that part to the "small things" area, works with it, and if necessary writes it back.
Some programmers gripe at having to distinguish between "near" and "far" memory, but in many cases making such distinctions can allow compilers to produce much better code.
(note: Even on many 32-bit systems, certain areas of memory can be accessed directly without extra instructions, while other areas cannot. If, for example, on a 68000 or an ARM, one keeps a register pointing at global variable storage, it will be possible to directly load any variable within the first 32K (68000) or 2K (ARM) of that register. Fetching a variable stored elsewhere will require an extra instruction to compute the address. Placing more frequently-used variables in the preferred regions and letting the compiler know would allow for more efficient code generation.
This terminology was used in 16 bit architectures.
In 16 bit systems, data was partitioned into 64Kb segments. Each loadable module (program file, dynamically loaded library etc) had an associated data segment - which could store up to 64Kb of data only.
A NEAR pointer was a pointer with 16 bit storage, and referred to data (only) in the current modules data segment.
16bit programs that had more than 64Kb of data as a requirement could access special allocators that would return a FAR pointer - which was a data segment id in the upper 16 bits, and a pointer into that data segment, in the lower 16 bits.
Yet larger programs would want to deal with more than 64Kb of contiguous data. A HUGE pointer looks exactly like a far pointer - it has 32bit storage - but the allocator has taken care to arrange a range of data segments, with consecutive IDs, so that by simply incrementing the data segment selector the next 64Kb chunk of data can be reached.
The underlying C and C++ language standards never really recognized these concepts officially in their memory models - all pointers in a C or C++ program are supposed to be the same size. So the NEAR, FAR and HUGE attributes were extensions provided by the various compiler vendors.
I understand the purpose of the NULL constant in C/C++, and I understand that it needs to be represented some way internally.
My question is: Is there some fundamental reason why the 0-address would be an invalid memory-location for an object in C/C++? Or are we in theory "wasting" one byte of memory due to this reservation?
The null pointer does not actually have to be 0. It's guaranteed in the C spec that when a constant 0 value is given in the context of a pointer it is treated as null by the compiler, however if you do
char *foo = (void *)1;
--foo;
// do something with foo
You will access the 0-address, not necessarily the null pointer. In most cases this happens to actually be the case, but it's not necessary, so we don't really have to waste that byte. Although, in the larger picture, if it isn't 0, it has to be something, so a byte is being wasted somewhere
Edit: Edited out the use of NULL due to the confusion in the comments. Also, the main message here is "null pointer != 0, and here's some C/pseudo code that shows the point I'm trying to make." Please don't actually try to compile this or worry about whether the types are proper; the meaning is clear.
This has nothing to do with wasting memory and more with memory organization.
When you work with the memory space, you have to assume that anything not directly "Belonging to you" is shared by the entire system or illegal for you to access. An address "belongs to you" if you have taken the address of something on the stack that is still on the stack, or if you have received it from a dynamic memory allocator and have not yet recycled it. Some OS calls will also provide you with legal areas.
In the good old days of real mode (e.g., DOS), all the beginning of the machine's address space was not meant to be written by user programs at all. Some of it even mapped to things like I/O.
For instance, writing to the address space at 0xB800 (fairly low) would actually let you capture the screen! Nothing was ever placed at address 0, and many memory controller would not let you access it, so it was a great choice for NULL. In fact, the memory controller on some PCs would have gone bonkers if you tried writing there.
Today the operating system protects you with a virtual address space. Nevertheless, no process is allowed to access addresses not allocated to it. Most of the addresses are not even mapped to an actual memory page, so accessing them will trigger a general protection fault or the equivalent in your operating system. This is why 0 is not wasted - even though all the processes on your machine "have an address 0", if they try to access it, it is not mapped anywhere.
There is no requirement that a null pointer be equal to the 0-address, it's just that most compilers implement it this way. It is perfectly possible to implement a null pointer by storing some other value and in fact some systems do this. The C99 specification §6.3.2.3 (Pointers) specifies only that an integer constant expression with the value 0 is a null pointer constant, but it does not say that a null pointer when converted to an integer has value 0.
An integer constant expression with the value 0, or such an expression cast to type
void *, is called a null pointer constant.
Any pointer type may be converted to an integer type. Except as previously specified, the
result is implementation-defined. If the result cannot be represented in the integer type,
the behavior is undefined. The result need not be in the range of values of any integer
type.
On some embedded systems the zero memory address is used for something addressable.
The zero address and the NULL pointer are not (necessarily) the same thing. Only a literal zero is a null pointer. In other words:
char* p = 0; // p is a null pointer
char* q = 1;
q--; // q is NOT necessarily a null pointer
Systems are free to represent the null pointer internally in any way they choose, and this representation may or may not "waste" a byte of memory by making the actual 0 address illegal. However, a compiler is required to convert a literal zero pointer into whatever the system's internal representation of NULL is. A pointer that comes to point to the zero address by some way other than being assigned a literal zero is not necessarily null.
Now, most systems do use 0 for NULL, but they don't have to.
It is not necessarily an illegal memory location. I have stored data by dereferencing a pointer to zero... it happens the datum was an interrupt vector being stored at the vector located at address zero.
By convention it is not normally used by application code since historically many systems had important system information starting at zero. It could be the boot rom or a vector table or even unused address space.
On many processors address zero is the reset vector, wherein lies the bootrom (BIOS on a PC), so you are unlikely to be storing anything at that physical address. On a processor with an MMU and a supporting OS, the physical and logical address addresses need not be the same, and the address zero may not be a valid logical address in the executing process context.
NULL is typically the zero address, but it is the zero address in your applications virtual address space. The virtual addresses that you use in most modern operating systems have exactly nothing to do with actual physical addresses, the OS maps from the virtual address space to the physical addresses for you. So, no, having the virtual address 0 representing NULL does not waste any memory.
Read up on virtual memory for a more involved discussion if you're curious.
I don't see the answers directly addressing what i think you were asking, so here goes:
Yes, at least 1 address value is "wasted" (made unavailable for use) because of the constant used for null. Whether it maps to 0 in linear map of process memory is not relevant.
And the reason that address won't be used for data storage is that you need that special status of the null pointer, to be able to distinguish from any other real pointer. Just like in the case of ASCIIZ strings (C-string, NUL-terminated), where the NUL character is designated as end of character string and cannot be used inside strings. Can you still use it inside? Yeah but that will mislead library functions as of where string ends.
I can think of at least one implementation of LISP i was learning, in which NIL (Lisp's null) was not 0, nor was it an invalid address but a real object. The reason was very clever - the standard required that CAR(NIL)=NIL and CDR(NIL)=NIL (Note: CAR(l) returns pointer to the head/first element of a list, where CDR(l) returns ptr to the tail/rest of the list.). So instead of adding if-checks in CAR and CDR whether the pointer is NIL - which will slow every call - they just allocated a CONS (think list) and assigned its head and tail to point to itself. There! - this way CAR and CDR will work and that address in memory won't be reused (because it is taken by the object devised as NIL)
ps. i just remembered that many-many years ago i read about some bug of Lattice-C that was related to NULL - must have been in the dark MS-DOS segmentation times, where you worked with separate code segment and data segment - so i remember there was an issue that it was possible for the first function from a linked library to have address 0, thus pointer to it will be considered invalid since ==NULL
But since modern operating systems can map the physical memory to logical memory addresses (or better: modern CPUs starting with the 386), not even a single byte is wasted.
As people already have pointed out, the bit representation of the NULL pointer has not to be the same as the bit represention of a 0 value. It is though in nearly all cases (the old dinosaur computers that had special addresses can be neglected) because a NULL pointer can also be used as a boolean and by using an integer (of suffisent size) to hold the pointer value it is easier to represent in the common ISAs of modern CPU. The code to handle it is then much more straight forward, thus less error prone.
You are correct in noting that the address space at 0 is not usable storate for your program. For a number of reasons a variety of systems do not consider this a valid address space for your program anyway.
Allowing any valid address to be used would require a null value flag for all pointers. This would exceed the overhead of the lost memory at address 0. It would also require additional code to check and see if the address were null or not, wasting memory and processor cycles.
Ideally, the address that NULL pointer is using (usually 0) should return an error on access. VAX/VMS never mapped a page to address 0 so following the NULL pointer would result in a failure.
The memory at that address is reserved for use by the operating system. 0 - 64k is reserved. 0 is used as a special value to indicate to developers "not a valid address".