diff options
| author | bors <bors@rust-lang.org> | 2016-05-15 05:26:50 -0700 |
|---|---|---|
| committer | bors <bors@rust-lang.org> | 2016-05-15 05:26:50 -0700 |
| commit | b3583193531c124e36aa1813cf5f8b8e667d016c (patch) | |
| tree | 6dc9e451e04dffefe9d51b9b587afd96cc88631d | |
| parent | 1a26d2364f9da1667ca0098d3cc4a213d131f481 (diff) | |
| parent | 04baf2593e782dafc979cfbfb6c9994e3a0d1e78 (diff) | |
Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michaelwoerister
gdb Pretty Print: generic encoded was failing on reference/pointer types
If you debug this program using **gdb**
```rust
fn main() {
let x = 10;
let y = Some(&x);
// additional code
}
```
And you try to print **y**'s value from the debugger, you get the following:
```
(gdb) print y
Python Exception <class 'gdb.error'> Cannot convert value to int.:
$1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}}
```
What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails.
Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration.
After this modification **gdb** prints **y** like this:
```
(gdb) print y
$1 = Some = {0x7fff5fbff97c}
```
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
| -rwxr-xr-x | src/etc/gdb_rust_pretty_printing.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/src/etc/gdb_rust_pretty_printing.py b/src/etc/gdb_rust_pretty_printing.py index f93f3490215..38c9fbf9828 100755 --- a/src/etc/gdb_rust_pretty_printing.py +++ b/src/etc/gdb_rust_pretty_printing.py @@ -70,6 +70,8 @@ class GdbValue(rustpp.Value): return child def as_integer(self): + if self.gdb_val.type.code == gdb.TYPE_CODE_PTR: + return int(str(self.gdb_val), 0) return int(self.gdb_val) def get_wrapped_value(self): |
