fix bigint unsigned null column scan to err type int64

[elonnzhang]

Description

when i use gorm scan sql rows to map[string]any, i got an error (out of range).
gorm scan sql.Rows to map[string]any by ScanRows
there has three step:

1. prepare values ​​according to column type. https://github.com/go-gorm/gorm/blob/master/scan.go#L25,

columnTypes, _ := rows.ColumnTypes()
prepareValues(values, db, columnTypes, columns)

2. rows.Scan(...any)

rows.Scan(values...)

3. assign column name and value to map

scanIntoMap(mapValue, values, columns)

in step 2, when occur number(tiny int short long longlong )null column, this type will be all set to scanTypeNullInt,

scanTypeNullInt    = reflect.TypeOf(sql.NullInt64{})

# "database/sql"#L210
// NullInt64 represents an int64 that may be null.
// NullInt64 implements the [Scanner] interface so
// it can be used as a scan destination, similar to [NullString].
type NullInt64 struct {
    Int64 int64
    Valid bool // Valid is true if Int64 is not NULL
}

tiny int short long type will be ok, its Maximum value will not out of range int64
but unsigned longlong may be out of range.
int64:-9223372036854775808 ~9223372036854775807,
uint64:0 ~ 18446744073709551615

a uint64 assign to NullInt64 will be out of range, because the value store into int64

so to fix it

Must add a new scan type NullUint64 to represent unsigned bigint null column.
and though flag to judge unsigned or not.

Dur to database/sql support generics Null[T] type, it will be easy to support NullUint64.

scanTypeNullUInt   = reflect.TypeOf(sql.Null[uint64]{})

Reproduce the problem

table example

CREATE TABLE `tbl_name` (
  `id` int NOT NULL COMMENT '',
  `ubigint` bigint unsigned DEFAULT NULL COMMENT '9223372036854775808',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `tbl_name` (`id`, `ubigint`) VALUES (1, 9223372036854775808);

code

func main() {
	// connect to db
	db, err := sql.Open("mysql", "dsn")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()

	gormDB, err := gorm.Open(gormMysql.New(gormMysql.Config{
		Conn: db,
	}), &gorm.Config{})
	var result map[string]any
	if err := gormDB.Raw("select * from tbl_name where id = 1").Scan(&result).Error; err != nil {
		log.Fatal(err)
	}
	// print type and value
	fmt.Printf("Type: %v\n", reflect.TypeOf(result))
	fmt.Printf("Value: %v\n", result)
}

Env

go version
go version go1.22.3 darwin/arm64

Error log

sql: Scan error on column index 1, name "ubigint": converting driver.Value type uint64 ("9223372036854775808") to a int64: value out of range
[9.744ms] [rows:1] select * from tbl_name where id = 1
2024/07/26 19:36:29 sql: Scan error on column index 1, name "ubigint": converting driver.Value type uint64 ("9223372036854775808") to a int64: value out of range

Checklist

• Code compiles correctly
• Created tests which fail without the change (if possible)
• All tests passing
• Extended the README / documentation, if necessary
• Added myself / the copyright holder to the AUTHORS file

Summary by CodeRabbit

• Documentation

  • Updated the AUTHORS file to include Jiabin Zhang as a contributor.

• Enhancements

  • Improved handling of unsigned integer types in the MySQL field representation, enhancing type safety for data scanning processes.

[coderabbitai]

Walkthrough

This update enhances the codebase by adding Jiabin Zhang as a contributor in the AUTHORS file, recognizing their contributions to the project. Additionally, a new reflection type, scanTypeNullUInt, is introduced in fields.go to improve the handling of unsigned integer types in MySQL fields. The scanType method in the mysqlField struct is modified to ensure better type safety during the processing of these unsigned integers.

Changes

Files	Change Summary
AUTHORS	Added Jiabin Zhang's name and email address to the authors list.
fields.go	Introduced scanTypeNullUInt for sql.Null[uint64]{}; modified scanType method to handle unsigned integers.

Poem

🐇 In fields of code where rabbits play,
New authors join, hip-hip-hooray!
With types anew, we hop with glee,
Reflecting all our unity!
A project blooms, together we thrive,
In every line, our spirit's alive! 🌼✨

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 44553d6 and 11f9287.

Files selected for processing (2)

• AUTHORS (1 hunks)
• fields.go (2 hunks)

Files skipped from review due to trivial changes (1)

• AUTHORS

Additional comments not posted (2)

fields.go (2)

131-131: LGTM!

The declaration of scanTypeNullUInt to handle sql.Null[uint64]{} is appropriate for addressing the issue of out-of-range errors when scanning uint64 values into NullInt64.

---

189-191: LGTM!

The modification to the scanType method to return scanTypeNullUInt when the flagUnsigned flag is set is correct. This ensures proper handling of unsigned bigint null columns.

[coveralls]

coverage: 82.364% (-0.04%) from 82.403%
when pulling 11f9287 on elonnzhang:fix_bigint_unsigned_scan_type
into 44553d6 on go-sql-driver:master.

[methane]

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

[elonnzhang]

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

ok，i just notice sql.Null introduced in 1.22

[coderabbitai]

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 11f9287 and 4a5269c.

Files selected for processing (1)

• fields.go (2 hunks)

Files skipped from review as they are similar to previous changes (1)

• fields.go

[elonnzhang]

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

Can we use sql.NullString to temporarily support all supported version avoid this problem (out of range)
until we support minimum version is go 1.22?

[methane]

I prefer 1 breaking change over 2 breaking changes.