Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Ticket #2016 (closed defect: fixed)

Opened 3 years ago

Last modified 2 years ago

PostgreSQL sequence_name and last_insert_id automatically detected with schema support

Reported by: robbyrussell Assigned to: David
Priority: normal Milestone: 1.x
Component: ActiveRecord Version: 0.13.1
Severity: normal Keywords: postgresql, sequence_name, schema, sql
Cc:

Description

Attached is a patch that will automatically detect the sequence name for any given schema.table in PostgreSQL. by default, it will assume public is the SCHEMA, but will also split on a schema.table_name and detect the proper sequence_name in the new method sequence_name, which performs the following SQL query against the PostgreSQL system tables.

            SELECT seq.relname::text
              FROM pg_class src, pg_class seq, pg_namespace, pg_attribute, pg_depend
              WHERE
                pg_depend.refobjsubid = pg_attribute.attnum AND
                pg_depend.refobjid = src.oid AND
                seq.oid = pg_depend.objid AND   
                src.relnamespace = pg_namespace.oid AND
                pg_attribute.attrelid = src.oid AND
                pg_namespace.nspname = 'SCHEMANAME' AND
                src.relname = 'TABLENAME' AND  
                pg_attribute.attname = 'PRIMARY KEY COLUMN';

This allows for full SCHEMA support such as:

CREATE SCHEMA foo;
CREATE TABLE foo.bars (id SERIAL, name VARCHAR(100));

The model would look like so:

class Foobar < ActiveRecord::Base
  def self.table_name() "foo.bars" end   
end

Within console:

$ ./script/console 
Loading development environment.
>> Foobar.create(:name => 'Robby Russell')
=> #<Foobar:0x275558c @new_record=false, @attributes={"name"=>"Robby Russell", "id"=>1}, @errors=#<ActiveRecord::Errors:0x2751fcc @base=#<Foobar:0x275558c ...>, @errors={}>>
>> Foobar.create(:name => 'Someone Else')
=> #<Foobar:0x274df6c @new_record=false, @attributes={"name"=>"Someone Else", "id"=>2}, @errors=#<ActiveRecord::Errors:0x274cbd0 @base=#<Foobar:0x274df6c ...>, @errors={}>>
>> Foobar.find:all
=> [#<Foobar:0x27495c0 @attributes={"name"=>"Robby Russell", "id"=>"1"}>, #<Foobar:0x2749584 @attributes={"name"=>"Someone Else", "id"=>"2"}>]
>>

Attachments

pg_adapter_sequence_name.patch_1.diff (2.3 kB) - added by robbyrussell on 08/20/05 01:26:28.
postgresql adapter patch with new sequence name and last_insert_id methods
index.rhtml (2.2 kB) - added by anonymous on 02/17/06 12:56:45.

Change History

08/20/05 01:26:28 changed by robbyrussell

  • attachment pg_adapter_sequence_name.patch_1.diff added.

postgresql adapter patch with new sequence name and last_insert_id methods

08/20/05 02:35:30 changed by robbyrussell

I need to stop trying to work on patches when I am sick or something. heh

Okay, I tested this patch a bit more and it broke on an uncommon sequence name such as in the following example:

testingdb=# \d legacy.foobar
                                     Table "legacy.foobar"
  Column   |         Type          |                         Modifiers                          
-----------+-----------------------+------------------------------------------------------------
 foobar_id | integer               | not null default nextval('legacy.old_sequence_name'::text)
 name      | character varying(40) | 
Indexes:
    "foobar_pkey" PRIMARY KEY, btree (foobar_id)
 
testingdb=# INSERT INTO legacy.foobar (name) VALUES ('abc')
testingdb-# ;
INSERT 17514 1
testingdb=# SELECT * FROM legacy.foobar ;
 foobar_id | name 
-----------+------
       106 | abc
(1 row)

The SQL query in the patch isn't consistent. I will have to keep checking. I did see this ticket #1273. I will test that out to see if it helps me figure out a way of handling a sequence name like above.

The one problem with #1273 is that it pretty much runs this SQL query:

# SELECT adsrc FROM pg_attrdef WHERE adrelid = (SELECT oid FROM pg_class WHERE relname = 'foobar');
                   adsrc                   
-------------------------------------------
 nextval('legacy.old_sequence_name'::text) 

In PostgreSQL, I can create a new schema, with the same table name and then the sub query would return multiple records, thus causing PostgreSQL to error.

foo=> CREATE SCHEMA x;
CREATE SCHEMA
foo=> CREATE TABLE x.foo (id SERIAL PRIMARY KEY NOT NULL, x TEXT);
NOTICE: CREATE TABLE will create implicit sequence "foo_id_seq" for
"serial" column "foo.id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
"foo_pkey" for table "foo"
CREATE TABLE
foo=> SELECT adsrc FROM pg_attrdef WHERE adrelid = (SELECT oid FROM
pg_class WHERE relname = 'foo');
ERROR: more than one row returned by a subquery used as an expression

So, it's obvious that the solution is not an easy one. I will keep digging around the PostgreSQL system tables and see if there is a way to provide a schema to one of these tables along with the table name. Passing *just* the table name is not going to be sufficient with that othat patch.

08/20/05 03:43:07 changed by robbyrussell

  • cc set to rails@bitsweat.net.
  • summary changed from [PATCH] PostgreSQL sequence_name and last_insert_id automatically detected with schema support to PostgreSQL sequence_name and last_insert_id automatically detected with schema support.

I am taking this out of [PATCH] for now. I think that this should be discussed a bit more. Perhaps Jeremy can enlighten me as to what he thinks should happen.

In my opinion, it would make more sense to rely on people using SERIAL rather than a custom named SEQUENCE. We should also be able to determine the sequence name. I need to be able to pass AR a custom sequence_name from time to time though to work with legacy databases and/or some custom sequence.

If my patch were applied (in terms of finding the sequence name with a schema, table name, and primary key column), it would be wise to cache the sequence name as it seems to be a rather complex ugly query. However, if we are going to make the assumption that a SEQUENCE is created by SERIAL is the standard, then we can just ignore this talk all together. ;-)

I think what we need is a way to pass AR a sequence name and not have it be created within the various methods as it is now.

class Foobar < ActiveRecord::Base
  def self.table_name() "legacy.foobar" end
  set_primary_key "foobar_id"
  set_sequence_name "legacy.old_sequence_name"
end

Or perhaps

  set_primary_key "legacy.foobar", :sequence => "legacy.old_sequence_name"

I'd prefer the latter method. A primary key has a sequence, and if we don't pass it one, it would just default to the standard sequence naming convention?

I would be happy to create a new patch that handled this approach and it would satisfy my needs for handling legacy schemas. :-)

08/20/05 03:49:55 changed by robbyrussell

Oops in the last example, it should be:

set_primary_key "foobar_id", :sequence => "legacy.old_sequence_name"

(not a big deal, but i had put the table name in there instead of the pkey column

08/22/05 09:54:15 changed by Ken Kunz <kennethkunz@gmail.com>

In a recent patch for the Oracle adapter (#1798), the following class methods were added to !ActiveRecord::Base:

  • sequence_name - defines default format for sequence name as "#{table_name}_seq"
  • set_sequence_name - used for overriding the default (also aliased as sequence_name=)

The Firebird Adapter (#1874) leverages this approach as well.

Wouldn't it make sense for PostgreSQL to use a consistent approach? The one problem I see is that the default sequence name above (for Oracle and Firebird) is inconsistent with the standard sequence names for PostgreSQL SERIAL columns.

This could be addressed by delegating the definition of the default sequence name format to the adapters. It's not ideal to have different sequence_name defaults for different databases, but I think it's even worse to have a single standard for AR that's inconsistent with common usage for a given RDBMS. Letting the adapter define the default sequence name seems like the lesser of two evils.

Thoughts? If folks agree with this approach, I'd be happy to write a quick patch.

08/22/05 14:40:10 changed by robbyrussell

Quick question. Is it ever possible that a table row might contain more than one SEQUENCE?

I'd think it would make more sense to declare the sequence name on set_primary_key, as it would be the primary keys sequence, correct?

08/22/05 22:51:15 changed by Ken Kunz <kennethkunz@gmail.com>

Robby,

I hear you... but I still think a separate method (set_sequence_name) makes more sense:

  • It's already implemented this way (#1798 - committed to trunk, thought not yet included in a release)
    • Situations were you'd have more than one sequence for a table are pretty rare. Rails doesn't support multi-column primary keys, so even if you used multiple sequences on a table, only one would be used for the PK (which is what this functionality is for -- auto-incrementing the PK value on insert). If you did have a need for multiple sequences on a table, you'd have to use a different mechanism for incrementing the ones that aren't tied to the PK (e.g., a BEFORE INSERT trigger).
    • Typically (for the reasons above), you can think of a sequence as being associated with a table. This is reflected in the common Oracle sequence naming convention of "table_name_seq".
  • In some cases, you might be using the default PK column name, but a custom sequence name; it seems more natural in these instances to just call set_sequence_name rather than have to call set_primary_key.
    • Several rdbms's supported by Rails don't use sequences; it seems less confusing to have a method that can be safely ignored by users of those db's (set_sequence_name), rather than altering the semantics of set_primary_key and having a new argument that they need to ignore.

Just my 2 cents.

08/28/05 16:08:59 changed by robbyrussell

Ken,

Okay, I think that I can live with this. I'll look over your patch and attempt to build a patch for PostgreSQL.

09/24/05 20:56:11 changed by rick

  • cc changed from rails@bitsweat.net to rails@bitsweat.net, technoweenie@gmail.com.

Hey guys, I took a stab at this. Let me know what you think.... #2292

02/17/06 12:56:45 changed by anonymous

  • attachment index.rhtml added.

04/05/06 16:30:21 changed by anonymous

Ticket #2292 is closed (thanks rick!). Does that mean we can safely close this ticket too?

07/20/06 11:35:30 changed by Barbara

  • severity changed from normal to 1.
  • cc changed from rails@bitsweat.net, technoweenie@gmail.com to Barbara.
  • component changed from ActiveRecord to 1.
  • summary changed from PostgreSQL sequence_name and last_insert_id automatically detected with schema support to Barbara.
  • priority changed from normal to 1.
  • version changed from 0.13.1 to 1.
  • milestone set to 1.
  • keywords changed from postgresql, sequence_name, schema, sql to Barbara.
  • type changed from defect to 1.

08/06/06 01:04:45 changed by anonymous

home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home home