Showing posts with label postgresql. Show all posts
Showing posts with label postgresql. Show all posts

Tuesday, December 30, 2008

Postgres in the Cloud

The Amazon AWS blog has a good story about about Soocial.com. Soocial.com is an address book on steroid. It works with (or will work with soon): outlook, blackberry, gmail, mac address book and other phones. They just need to add Yahoo and Windows Contacts.

I wrote about it on my cloud blog, 100% AWS.

The AWS blog post has a good overview of the overall architecture. They are using Postgres and pgpool II on EC2 and EBS. They use RabbitMQ for messaging. For high availability they are using HAProxy and memcache.

It sounds like a pretty sweet setup.

LewisC

Technorati : , , , , ,

Wednesday, July 23, 2008

Update on EDB Open Source Database Survey

Yesterday I posted about the results of the EnterpriseDB open source survey. In that post, I said:

I'd like to see the survey again and compare the results to the survey itself.

I discovered that, as of right now, the survey is still online.

Still no information as to when, or if, the entire survey results will be released. That's what I am most interested in.

LewisC

Technorati : , , ,

Saturday, July 19, 2008

Not working for EDB anymore

Well, I am no longer working for EnterpriseDB. It was fun while it lasted but it's over so I am moving on. I found a new job, locally. It's pure Oracle and I will get to use Real Application Clusters in a production environment. That's something I haven't done in the past so I am looking forward to it. It's also a java, .net and Oracle Forms shop and they are doing some interesting things with telecommunications and SMS.

From now on, I will only need to travel for conferences. No more trips to New Jersey. That's kind of a drag as I was almost at elite status on Continental. I have two more conferences this year and it just might be enough. I may even take a trip on my own just to get the miles. One of the trips is from Tampa to San Francisco, and back, so that will get me very close. The other is to Virginia and back.

I am a database geek and I will be keeping an eye on EDB just as I did in the past. I will probably post here at the EDB blog about as frequently as I have been (not often). Or, I may let this one die and just do any EDB posting on my postgres blog. That actually makes the most sense. I think I will cross post this one there and make this my last dedicated posting on this blog.

I also plan to keep up with Postgres, for personal knowledge, just as I do MySQL. I actually want to install GridSQL and see how it performs for a variety of different applications.

On the upside, I can now call EnterpriseDB Postgres Plus Advanced Server, PP AS, without marketing having fit. ;-) Heck, if I think of it as Advanced Server Software, I can call it the PP ASS. heh But I would never do that.

So, I can guess I can close out this blog now. Later.

LewisC

Technorati : , ,

Tuesday, December 25, 2007

New Postgres Online Magazine

I ran across a new Postgres ezine, Postgres Online Journal. Dec 2007 is complete and Jan 2008 is being constructed. You can read the current, under construction issue as HTML while it is being constructed. It looks like you can also download a PDF when an issue is complete. It will be nice to have an additional resource for Postgres developers and DBAs. This is just a sign that Postgres is becoming more mainstream. In the past all online Postgres information came from a very small group of people. The Dec 2007 issue has some good info:

  • PostgreSQL The Road Behind and Ahead
  • PostgreSQL 8.3 is just around the Corner
  • Converting from Unix Timestamp to PostgreSQL Timestamp or Date Beginner
  • Using Distinct ON to return newest order for each customer Intermediate
  • How to create an index based on a function Intermediate
  • The Anatomy of a PostgreSQL - Part 1 Beginner
  • How does CLUSTER ON improve index performance Intermediate
  • Language Architecture in PostgreSQL Intermediate
  • PostGIS for geospatial analysis and mapping Intermediate
  • Database Abstraction with Updateable Views Advanced
The article on updateable views is very good. I have used updateable views A LOT in Oracle but hadn't given that any thought in Postgres. DISTINCT ON is a nice feature also. You can click on About The Authors to see who is writing this. It leads to a company site called Paragon Corporation. They appear to be a database consulting company that works with most databases. This is a nice addition to the Postgres online resource pool. LewisC

Sunday, October 7, 2007

Best Way to Handle No Data Found in a Procedure?

When it comes to data issues (too many rows, no data found, etc), in Oracle stored procedures, I am used to having exceptions raised that I then handle. PL/pgSQL does not raise exceptions for the same conditions in the same way. The Postgres docs are pretty complete though and through some reading this weekend, I discovered a new keyword. For example, assuming that I have this table (which is empty) in both Oracle and Postgres:

CREATE TABLE empty_table
(
  empty_col integer
);
In Oracle this procedure:
CREATE OR REPLACE PROCEDURE no_data_found_test
AS
  v_int_field INTEGER;
BEGIN

  SELECT empty_col
    INTO v_int_field
    FROM empty_table;

END;
When run:
BEGIN
  no_data_found_test;
END;
Produces an error:
Error starting at line 1 in command:
BEGIN
  no_data_found_test;
END;
Error report:
ORA-01403: no data found
ORA-06512: at "HR.NO_DATA_FOUND_TEST", line 6
ORA-06512: at line 2
01403. 00000 -  "no data found"
*Cause:    
*Action:
However, the equivalent procedure in Postgres:
CREATE OR REPLACE FUNCTION no_data_found_test()
  RETURNS void AS
$BODY$
DECLARE
  v_int_field INTEGER;
BEGIN

  SELECT empty_col
    INTO v_int_field
    FROM empty_table;

END;
$BODY$
  LANGUAGE 'plpgsql' VOLATILE;
Does not raise an exception when run:
postgres=# select * from no_data_found_test();
 no_data_found_test
--------------------

(1 row)
I have been using the FOUND variable to check for a result and raise an exception manually if no data was found. Like this:
CREATE OR REPLACE FUNCTION no_data_found_test()
  RETURNS void AS
$BODY$
DECLARE
  v_int_field INTEGER;
BEGIN

  SELECT empty_col
    INTO v_int_field
    FROM empty_table;

  IF NOT FOUND THEN
    raise exception 'NO_DATA_FOUND';
  END IF;

END;
$BODY$
  LANGUAGE 'plpgsql' VOLATILE;
When run, I do get an exception like I was expecting.
postgres=# select * from no_data_found_test();
ERROR:  NO_DATA_FOUND
However, after a little bit more reading, I see that I can add the STRICT keyword to make the procedure behave like Oracle:
CREATE OR REPLACE FUNCTION no_data_found_test()
  RETURNS void AS
$BODY$
DECLARE
  v_int_field INTEGER;
BEGIN

  SELECT empty_col
    INTO STRICT v_int_field
    FROM empty_table;

END;
$BODY$
  LANGUAGE 'plpgsql' VOLATILE;
And now when run, it does raise the exception:
postgres=# select * from no_data_found_test();
ERROR:  query returned no rows
CONTEXT:  PL/pgSQL function "no_data_found_test" line 5 at SQL statement
postgres=#
Very Nice! I think I prefer this method. It would be nice if STRICT were a database wide configuration parameter. LewisC

Friday, October 5, 2007

Postgres LiveCD

I read about the Postgres LiveCD at the Blue Gnu. That's such a good idea! I don't know why we don't see more LiveCDs. Even commercial software should be taking advantage of such an easy marketing opportunity. The entry says:

Well, what can I say? The CD is simply Xubuntu with PostgreSQL loaded up and ready to run. And once loaded, you can actually play with PostgreSQL at least three different ways. The Xubuntu CD offers three icons that launch the Command-Line client, PGAdmin3 or phpPGAdmin. Now you can play with the server for as long as you like, goof it up and reboot for a clean, shiny new server. Actually, the CD includes some sample databases and Slony-I, the database replication utility. Now, I didn't notice any documentation lying around, but my guess is one could probably mount the local hard drive and have PostgreSQL create databases on it all day long without ever installing the actual database server. Xubuntu still contains (most of) the other applications normally included - the team has simply loaded PG and it's client applications.
I couldn't find any links about where to get an ISO or anything so I went searching. I found some email list messages from 2006 and two interesting links: As a bonus, Dru links to Joshua Drake's Practical PostgreSQL. This free book is getting a little old but still makes a useful reference. LewisC

Saturday, September 15, 2007

Programming Postgres

This is just a quick overview of the availability of programming for Postgres. With Oracle, your choice in the database is Java or PL/SQL. In general, PL/SQL is the preferred language. One of the things that attracted me to Postgres was the ability to choose one of many languages to program in. I usually choose PL/pgSQL but if I can't do something with that, it's nice to be able to fall back to TCL, Perl or one of the many other languages supported by Postgres. Here is a (incomplete, I'm sure) list of supported languages:

  • PL/pgSQL - Standard Procedural Language for Postgres
  • PL/TCL - TCL, Tool Command Language, easy to learn but powerful
  • PL/Perl - Practical Extraction and Report Language, has grown way beyond its origins
  • PL/Java - Java, meh
  • PL/PHP - PHP: Hypertext Preprocessor, over hyped language
  • PL/Python - Object Oriented, dynamic language. Almost as over hyped as PHP
  • PL/R - Statistical Language, never used it.
  • PL/Ruby - Great language. Over hyped maybe but is my current favorite open source language
  • PL/Scheme - Lispy language, never used it
  • PL/sh - Access to your OS shell language
Adding a new language to your existing database is very easy. You can get the details from the Postgres CREATE LANGUAGE syntax docs. The way you will most often add a language is: CREATE LANGUAGE For example: CREATE LANGUAGE pltcl; The language must already be installed in the OS. You will use the dynamic libraries from the language to create and run your programs. You also need the Postgres component for your language. PL/pgSQL, PL/TCL and PL/Perl come with the standard distribution of Postgres. You can get other languages at http://pgfoundry.org/. Run a search on pgFoundry to see what they have.

Sunday, September 9, 2007

Foxpro to Postgres Data Converter

Do you remember Foxpro? xBase? Clipper? I'm an old clipper head from way back. Anyway, I ran across something in my recent internet travels that caught my fancy. It may have the longest name of any utility I have ever seen. FoxPro PostgreSQL Import, Export & Convert Software will transfer tables to and from PostgreSQL and FoxPro. I haven't downloaded it yet so I can say how well it works but it's a neat idea. I'm guessing it can handle DBF files so that would make it compatible with dBase, Foxpro, Clipper and may other tools. It's a shareware program and I don't see a price. It's been a while since I even ran across shareare. That's a flashback for me too. I use to write and sell shareware. Anyway, here's the blurb for it:

This application will allow you to transfer tables to and from PostgreSQL and FoxPro databases. Importing and exporting can be done without SQL knowledge. This program saves time if you need to transfer tables between the two databases quickly. Connect via DBC file or multiple DBF files in a folder.
Take care, LewisC

Friday, September 7, 2007

Calling a Procedure or Function in Postgres

I think PL/SQL programmers who move to Postgres all run into the same thing, how do I run the procedure or function once I've created it?

Obviously, it's easy when you are calling it from another stored procedure or function.  What most mean, is how do I call it from the command line?  That's easy too.

Using my function and procedure from earlier posts, i.e. 10 Steps to Creating a Function In PostgreSQL Using PLpgSQL and Creating a Procedure In PostgreSQL Using PLpgSQL.

To call either the function or the procedure, you can run:

SELECT func_1();

Or

SELECT * FROM func_1();

You must include the ().

Same for the procedure.

SELECT proc_1();

Or

SELECT * FROM proc_1();

Of course, if you are using a GUI tool like PgAdmin III, it's a somewhat different story.  You need to open the query window but the syntax will be the same.

Technorati Tags: , , , , , , , ,

Wednesday, September 5, 2007

Creating a Procedure In PostgreSQL Using PLpgSQL

I recently wrote about creating a function in PL/pgSQL, 10 Steps to Creating a Function In PostgreSQL Using PLpgSQL.

Today, I am going to show you how to create a procedure.  You don't really create a procedure, you use the same basic syntax as you do for a function.  The RETURNS keyword can get a little tricky as that keyword requirement changes based on what you are trying to do.  To start, I will create a procedure that functions the same as the function in the previous entry.  Then I will make some changes.

CREATE OR REPLACE FUNCTION proc_1(
  OUT out_parameter CHAR VARYING(25) )
AS $$
DECLARE
BEGIN
  SELECT datname
    INTO out_parameter
    FROM pg_database
    LIMIT 1;

  RETURN;
END;
$$ LANGUAGE plpgsql;

If you compare this to the function from the previous entry some things jump out right away.  There is no RETURNS statement.  Postgres is able to determine the return type from the OUT parameter.

The DECLARE keyword is completely optional.

The RETURN keyword has no operand.  I don't need to RETURN a variable because the OUT parameter will be the return value.  As a matter of a fact, if I try to RETURN a value, I will get an error.

Below is another procedure that does basically the same thing but does not return a value.  I am showing you this just to show you how to declare a procedure that truly acts as a procedure (that does not return any values).

CREATE OR REPLACE FUNCTION proc_3()
RETURNS void
AS $$
DECLARE
  local_char_var CHAR(30);
BEGIN
  SELECT datname
    INTO local_char_var
    FROM pg_database
    LIMIT 1;

  RETURN;
END;
$$ LANGUAGE plpgsql;

Notice in this procedure that I am declare the return value as RETURNS void.  If you are a C, Java or C# programmer this should be very familiar.

For a PL/SQL programmer, just consider that a FUNCTION RETURNS VOID is the same as a PROCEDURE in PL/SQL.

That's it for this post.  If you have anything specific that you would like me to cover, please leave a comment or drop me an email.

Thanks,

LewisC

 

Monday, September 3, 2007

Should PostgreSQL be renamed and of so, to what?

For, I guess, about the last week or so, there has been an ongoing debate on the Advocacy email list about renaming the software from PostgreSQL to Postgres or something else.

Some people have very strong opinions about it. Personally, I don't see what the big deal is. The only people who really have an issue are the people in that (hacker) community. CEOs don't care what it's called. No developer I know cares what it's called.

Just to clarify, when I say developer, I mean a person who develops software for business use. Not someone who actually writes code for the database itself (those are the hackers).

So what I mean, is that the guy writing a GL, using PostgreSQL, doesn't care if it's PostgreSQL, Postgres, Postres QL, pg, pgsql or anything else. They just want the software to work.

At the beginning of the discussion on the mail list someone complained about people mis-pronouncing the name, PostgreSQL. Que? How can you mis-pronounce a made up name? It's an imaginary word. If I want to pronounce it "frank" am I wrong? Maybe in your universe but not in mine.

Many people have suggested to just say that Postgres is the preferred spelling (and pronunciation) and PostgreSQL is less preferred but acceptable spelling. I would have to agree with that one. If the pronunciation really bothers someone, let them write a blog about it. C'est la vie.

LewisC


Sunday, September 2, 2007

10 Steps to Creating a Function In PostgreSQL Using PLpgSQL

It's actually fairly easy to create a function using PLpgSQL, especially if you are coming from a database background like Oracle or DB2. Both have procedural languages that look a lot like PLpgSQL. I'll go ahead and show you the code for a very basic function and then I'll explain the steps individually.

CREATE OR REPLACE FUNCTION func_1()
  RETURNS CHAR VARYING(25) 
AS $$
DECLARE
  local_char_var CHAR(30);
BEGIN
  SELECT datname
    INTO local_char_var
    FROM pg_database
    LIMIT 1;

  RETURN local_char_var;
END;
$$ LANGUAGE plpgsql;
Ok. Now we'll go through it line by line:
  1. CREATE OR REPLACE FUNCTION func_1() - This line creates and names the function. The "OR REPLACE" will let us modify the function without fir DROPping it. func_1 can be any valid PLpgSQL name. Even if you are not declaring parameters, you must include the parenthesis ().
  2. RETURNS CHAR VARYING(25) - RETURNS is the keyword that signifies what data type the function will be returing. RETURN is the equivalent in PL/SQL. Notice the S. CHAR VARYING is the equivalent of a VARCHAR2 in PL/SQL. In this case it will be a VARCHAR2(25).
  3. AS $$ - AS is the same as the PL/SQL AS. The $$ is the function code delimiter. You can actually use single quotes here instead of $$ but in that case you would need to double all the quotes in your code. If that doesn't make sense, just consider the $$ mandatory. PLpgSQL is a more basic language than PL/SQL. The code is stored as text and compiled as it is run. The entire function body is just a string and the $$ is the delimiter.
  4. DECLARE - used to begin the variable declaration area. If you are not declaring local variables, this is an optional keyword.
  5. local_char_var CHAR(30); - A locally declared variable.
  6. BEGIN - Begins the body of the the function.
  7. SELECT datname INTO local_char_var FROM pg_database LIMIT 1; - This select statement is selecting the database name from the PG_DATABASE data dictionary table. The database name is being stored in the local variable local_char_var. The query is limiting the result set to 1 row.
  8. RETURN local_char_var; - This line returns the local variable back to the calling program.
  9. END; - Ends the body of the function.
  10. $$ LANGUAGE plpgsql; - The $$ is the end delimiter of the function body (think of it as ending the string). The LANGUAGE plpgsql identifies the language type to the postgres engine so that it knows which language to run the program.
That's it for a very basic function. I'll build on this in the future. LewisC

Friday, August 31, 2007

PostgreSQL 8.3 Features To Be Available After Labor Day

InformationWeek has an article with Bruce Momjian, PostgreSQL 8.3 Features To Be Available After Labor Day: Full text search and other features will become available for free download in beta code after Labor Day; with the final release to follow by 8-10 weeks. Won't be long now. I probably won't have time to play with it while it's in beta but I am looking forward to it. Tsearch2 will be nice and compares to Oracle Text although not quite a feature rich. For some reason, EnterpriseDB's PL/SQL debugger is being included. That doesn't make a lot of sense to me as they are not releasing SPL (their version of PL/SQL) to be included. I guess it's being included because it also supports PL/pgSQL (the postgres procedural language). Bitmap indexes are not making it into this release. That would have been nice for larger installations like data warehouses. Not mentioned in the article but will be there is SQL/X support. I am a huge user of XML and SQL/X makes life so much easier when generating XML. I wonder when XQuery will be supported?

Tuesday, August 28, 2007

Nice Little Cheat Sheet

I stumbled across a nice little cheat sheet by someone names Pete Freitag. It's a PostgreSQL Cheat Sheet. The sheet contains a quick ref for CREATE DATABASE, CREATE TABLE, Adding a primary key, Creating an INDEX, Backing up a database, Running a SQL script, Selecting using a regular expression, Selecting the first N records, Using Prepared Statements, Creating a Function, Vacuum, Reindexing, Showing a query plan, Importing from a file, and some basic SQL statements. It's not a bad quick ref for newbies. The site also has cheat sheets for Coldfusion, Java, Apache, ASCII codes, CSS, XHTML, and even an English Grammer Cheat Sheet. The author also includes a pretty long list of cheat sheets available on other sites. LewisC

Monday, August 27, 2007

What is included in EnterpriseDB Postgres?

I have had the question several times now so I thought I would blog about. This info was actually covered in the original press release, EnterpriseDB Announces First-Ever Professional-Grade PostgreSQL Distribution for Linux. Professional-Grade PostgreSQL Distribution EnterpriseDB Postgres is an open source, pre-configured, certified binary PostgreSQL distribution that simplifies enterprise deployment, eliminating the need to source multiple software components on the Web and manually assemble them. EnterpriseDB Postgres includes a one-click installer that delivers the most commonly needed PostgreSQL components and add-ons for major operating systems, including:

  • The most recent General Availability version of PostgreSQL, plus:
    • Secure Sockets Layer (SSL)
    • Cryptography (pgCrypto)
    • XML Support (libxml)
    • Full Text Search (TSearch2)
    • Database Linking (DBLink)
    • Languages: pl/pgSQL, pl/TCL, pl/Perl
    • Database Connectors: ODBC, JDBC
  • Graphical administration and monitoring (pgAdmin III and phpPgAdmin)
  • Replication (Slony-I)
  • Geospatial information server (PostGIS)
  • Comprehensive documentation
They are also offering support, documentation and forums. LewisC

Friday, August 24, 2007

Create a Partitioned Table In PostgreSQL

First, we create the table structure that will act as the master table.


CREATE TABLE sales_range 
(salesman_id  integer, 
salesman_name text, 
sales_amount  integer, 
sales_date    timestamp);
Then we create each of the partitioned tables using OO style syntax. Each partitioned table "inherits" the structure of the parent table. There is a string relation between parent and child. You cannot drop the parent without dropping the child.

CREATE TABLE sales_jan2000
  ( CHECK (sales_date < TO_DATE('02/01/2000','MM/DD/YYYY')) )
  INHERITS (sales_range);

CREATE TABLE sales_feb2000
  ( CHECK (sales_date < TO_DATE('03/01/2000','MM/DD/YYYY')) )
  INHERITS (sales_range);

CREATE TABLE sales_mar2000
  ( CHECK (sales_date < TO_DATE('04/01/2000','MM/DD/YYYY')) )
  INHERITS (sales_range);

CREATE TABLE sales_apr2000
  ( CHECK (sales_date < TO_DATE('05/01/2000','MM/DD/YYYY')) )
  INHERITS (sales_range);
At this point, any inserts against the parent table (sales_range) would be inserted into sales_range. To actually use the new partitions, we need to write either a PostgreSQL Rule or a trigger. I like triggers personally. I'll create the trigger function. I'll have a separate IF for each type of operation (tg_op) and an IF for each partition within that operation. You can simplify this with dynamic SQL but I will save that for a future posting.

CREATE FUNCTION sales_range_handler() 
  RETURNS TRIGGER AS $$
DECLARE
BEGIN
  IF tg_op = 'INSERT' THEN
    IF new.sales_date < cast('2000-02-01' as timestamp)
    THEN
      INSERT INTO sales_jan2000
         (salesman_id, salesman_name, sales_amount, sales_date)
        VALUES (new.salesman_id, new.salesman_name, 
                new.sales_amount, new.sales_date);
    ELSIF new.sales_date < cast('2000-03-01' as timestamp)
    THEN
      INSERT INTO sales_feb2000
         (salesman_id, salesman_name, sales_amount, sales_date)
        VALUES (new.salesman_id, new.salesman_name, 
                new.sales_amount, new.sales_date);
    ELSIF new.sales_date < cast('2000-04-01' as timestamp)
    THEN
      INSERT INTO sales_mar2000
         (salesman_id, salesman_name, sales_amount, sales_date)
        VALUES (new.salesman_id, new.salesman_name, 
                new.sales_amount, new.sales_date);
    ELSIF new.sales_date < cast('2000-05-01' as timestamp)
    THEN
      INSERT INTO sales_apr2000
         (salesman_id, salesman_name, sales_amount, sales_date)
        VALUES (new.salesman_id, new.salesman_name, 
                new.sales_amount, new.sales_date);
    END IF;
  ELSIF tg_op = 'UPDATE' THEN
    -- Do the same for update
    NULL;
  ELSIF tg_op = 'DELETE' THEN
     -- Do the same for delete
    NULL;
  END IF;
  RETURN NULL;
END;
$$   LANGUAGE plpgsql;
Then I create the actual trigger:
CREATE TRIGGER sales_range_handler_trg 
  BEFORE INSERT OR UPDATE OR DELETE 
  ON sales_range
  FOR EACH ROW 
  EXECUTE PROCEDURE sales_range_handler();
Finally, I do an INSERT:
INSERT INTO sales_range (salesman_id, salesman_name, sales_amount, sales_date)
  VALUES (1, 'Lewis', 10.25, cast('2000-03-15' as timestamp) );
If you select from the main table, you will see one record. That record is actually being selected from the MAR2000 table. You can select directly from that table and see that the record is there. To optimize your partitioned queries, check out Constraint Exclusion. LewisC

Tuesday, August 21, 2007

10 PostgreSQL versus Everything Else Comparisons

Feature comparisons, in addition to being somewhat lame and almost always biased, are very dependent on versions. If you throw in performance considerations, they are also dependent on hardware and configuration. Even so, I like to read comparisons, for historical information if nothing else. Here are some comparisons that I have found. I make no recommendations or even commentary about them. Read through them as if you are a database anthropologist. Dig for the nuggets that increase your existing base of knowledge but remember that a human, probably one with an agenda, put these comparisons together. I have also found that PostgreSQL is most often compared to MySQL and not to one of the large commercial databases such as Oracle or DB2. Update: August 27. A new comparison: PostgreSQL vs Firebird, August 2007 (and hopefully it will remain updated)

  1. PostgreSQL or MySQL - Feb 15 2005, a fairly nice comparison actually.
  2. PostgreSQL vs. MySQL vs. Commercial Databases: It's All About What You Need - DevX April 12, 2004
  3. PostgreSQL vs MySQL: Which is better?, DatabaseJournal Dec 16, 2003
  4. PostgreSQL vs. SQL Server: PostgreSQL is right for the Microsoft stack, SearchEntepriseLinux May 15, 2006
  5. PostgreSQL vs. Oracle: Users speak out, SeachOracle April 6, 2006
  6. PostgreSQL vs MySQL with Rails, June 18, 2005
  7. Oracle 10g vs PostgreSQL 8 vs MySQL 5, This one I wrote, Aug 22, 2005
  8. Firebird vs Postgres, Forum Post, April 23, 2002
  9. PostgreSQL vs. MySQL (Web Techniques, Sep 2001), Dr Dobbs, Jan 1, 2002
  10. MySQL vs. PostgreSQL, Aug 9, 2005
Enjoy, LewisC

Friday, August 17, 2007

There is a New PostgreSQL Blog Aggregator In Town

I used Yahoo Pipes to create a new PostgreSQL blog aggregator. I am hosting it on squidoo. Check out the Squidoo All About PostgreSQL page. You can vote for your favorite PostgreSQL book, leave comments and check out postgres and database related products. BTW, this is the first I have use Yahoo Pipes. It's very cool and very easy to use. It took me about 10 minutes to use it and about 5 minutes to set up my aggregator. If you have ever used a graphical ETL tool, it looks a lot like that. Take care and check out the Squidoo All About PostgreSQL page! LewisC

Wednesday, August 15, 2007

Learn PostgreSQL: Newbie Questions

A poster on the PostgreSQL pgsql-general mail list asked some good questions that might be important to anyone. The general mail is a:

General discussion area for users. Apart from compile, acceptance test, and bug problems, most new users will probably only be interested in this mailing list (unless they want to contribute to development or documentation). All non-bug related questions regarding PostgreSQL's version of SQL, and all installation related questions that do not involve bugs or failed compiles, should be restricted to this area. Please note that many of the developers monitor this area.

The poster says he is a long time MySQL user and was switching to PostgreSQL. He had several reasons but sumed it up as, "...the true features included in 5.1 (as of this moment) are nothing to write home about. The InnoDB stuff is highly advocated but it has its own set of issues,...".

So he was posting some questions to make sure he was heading in the right direction. You can read the actual email here. I am going to sum up some of the emails for you below but if these questions interest you, I would suggest reading the entire thread.

The poster asked four questions but for this entry, I am going to concentrate on his first question. I may add additional entries later to cover his other questions.

His first question regarded performance on DML around a hot table. 10k inserts/day, 500k selects/day and 1M updates/day. He had heard the wrapping the statements in a BEGIN TRANSACTION; and COMMIT; would increase the performance. This generated many great responses.

Here is a partial list of comments and considerations from this question. Many of these I knew but there were a few nuggets that were new to me. Some of these are exact copies of responses and some are slightly paraphrased by me, but the knowledge is actually coming from others.

  • A transaction is a bunch of queries which you want to all get committed or aborted together. The expensive step in Postgres is the COMMIT. Postgres does an fsync which forces the data to go to disk (if you're using good hardware, and don't have fsync=off). That takes from 5-15ms depending on how much data and how fast your drives are.
  • Grouping more work into a single transaction makes the delay for the fsync at COMMIT time less of a problem. Also having more connections (but not too many, more than a few per processor is probably not helping, more than 100 and it's probably slowing you down) also means it's less important since another process can do some of its work while you're waiting for the fsync.
  • You want "fsync=on" on any machine which holds data you care about. And you want hardware which doesn't lie to you so that "fsync is finished" really means the data is on-disk. Else PostgreSQL cannot ensure ACID compliance.
  • psql runs in autocommit mode by default. If you want multiple queries in a transaction you have to issue a BEGIN statement. Drivers may do various things by default.
  • The rule is, if any query within the transaction errors, then all queries within the transaction are rolled back.
  • Build your application around the application needs first, then later look at how to optimize it.
  • Remember the two rules of optimization: 1) Don't 2) (for experts only) Don't yet
  • The only case where you should consider batching together transactions like that is if you're processing a batch data load of some kind. In that case you have a large volume of updates and they're all single-threaded. But usually in that case you want to abort the whole load if you have a problem.

All in all some really good info. If you are picking up Postgres for the first time, I would suggest you sign up for the general mail list and lurk for a while or just head over to the link above and browse.

LewisC

Sunday, August 12, 2007

THE Postgres Resource Center

Where do you go when you have a question about Postgres? Do you join one of the PostgreSQL.org mail lists? That is so 1990s, isn't it? Not that I would recommend signing up for a postgres twitter but how about checking out the EnterpriseDB Postgres Resource Center. You can get news, downloads, documentation and a support forum. The blurb on the site says:

The EnterpriseDB Postgres Resource Center is a community-based site for enterprise application developers and DBAs and provides a rich repository of technical information, tools, and other resources. The site also hosts community-based forums, enabling EnterpriseDB Postgres users to interact with peers and leverage the collective experience of the EnterpriseDB Postgres community.
The forums are still fairly new so there isn't a lot of traffic yet, but that will come with time. There is a forum for news, postgres beginners, postgres on windows, postgres on linux and postgres on mac. Check it out. LewisC

PostgreSQL Books