Thursday, August 17, 2017

Referencing package-level variables inside the package body

I received this question yesterday:
Is there a shortcut for referencing package variables in the package body? In Java, as an example, you can use the "this" keyword as a reference to the current object. This came about as I needed to create a copy of a package for debug purposes and realized I had to rename all the references to the package name within the package body.
Before I answer, let's look at an example of what Tony is talking about. I create a package specification and then a body with package-level variable (declared outside any subprogram of the package):

CREATE OR REPLACE PACKAGE pkg
   AUTHID DEFINER
IS
   PROCEDURE proc (n_in IN NUMBER);
END;
/

CREATE OR REPLACE PACKAGE BODY pkg
IS
   smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
   BEGIN
      IF n_in < pkg.smallest
      THEN
         DBMS_OUTPUT.put_line ('too small');
      END IF;
   END;
END;
/

Notice the line in blue and bold. I reference the package level variable, qualified with the package name (the approach that Tony has taken).

But then when Tony changes the name of the package, so he also has to change the dot-qualified references within the package.

Can you use "this" syntax, as is available in Java? No. You will see this error when trying to compile the package body:

PLS-00201: identifier 'THIS.SMALLEST' must be declared

"this"-style syntax is used only with object types in Oracle Database (our object-oriented extensions to the relational model), in which case you use "SELF" to reference the current object type instance.

So what should Tony do? Here are his (and mine, and your) options:
  • Continue to dot qualify with the package name, and do a (careful, very careful) global search and replace of the old package name to new. Like I say: CAREFUL.
  • Remove the package name. You don't need it.
That's right. You don't need to qualify the reference to this variable. In other words, this version of he package body will compile just fine:

CREATE OR REPLACE PACKAGE BODY pkg
IS
   smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
   BEGIN
      IF n_in < smallest
      THEN
         DBMS_OUTPUT.put_line ('too small');
      END IF;
   END;
END;
/

The compiler will try to resolve the reference to smallest inside proc. When that doesn't work, it will check the "next level up" in scope: the package body. Ah ha! There it is. And the body compiles.

Of course, this will not work as desired if you declare a variable or constant with the same name inside proc. The following version will also compile, but that reference to smaller inside proc will not refer to the package level variable.

CREATE OR REPLACE PACKAGE BODY pkg
IS
   smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
      smallest NUMBER;
   BEGIN
      IF n_in < smallest
      THEN
         DBMS_OUTPUT.put_line ('too small');
      END IF;
   END;
END;
/

In this case, you must qualify the variable with the scope you mean. You could qualify with the subprogram (procedure) or package name, by the way:

CREATE OR REPLACE PACKAGE BODY pkg
IS
   smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
      smallest NUMBER;
   BEGIN
      IF n_in < proc.smallest -- references procedure's smallest
      THEN

      IF n_in < pkg.smallest -- references package level smallest
      THEN
         DBMS_OUTPUT.put_line ('too small');
      END IF;
   END;
END;
/

Of course, best of all is to avoid declaring variables with the same names at different scopes. That can be confusing - and then requires the use of dot-qualification. Another approach, frequently used, is to use different naming conventions for variables of different scope. For example, I commonly use "g_" to indicate a global variable in a package, and "l_" for local variables.

So my version of the package body would look like:

CREATE OR REPLACE PACKAGE BODY pkg
IS
   g_smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
      l_smallest NUMBER;
   BEGIN
      IF n_in < l_smallest -- references procedure's smallest
      THEN

      IF n_in < g_smallest -- references package level smallest
      THEN
         DBMS_OUTPUT.put_line ('too small');
      END IF;
   END;
END;
/

The most important thing is to be consistent in your approach so others (including the Future You) can more easily understand your code.

A final comment regarding dot-qualifying variable names: when you reference PL/SQL variables and constants inside SQL statements in your PL/SQL blocks, you should always qualify them with their scope names. Example:

CREATE OR REPLACE PACKAGE BODY pkg
IS
   g_smallest   NUMBER;

   PROCEDURE proc (n_in IN NUMBER)
   IS
      l_salary NUMBER;
   BEGIN
      SELECT salary INTO l_salary
        FROM employees e
       WHERE e.employee_id = proc.n_in;
   END;
END;
/

By doing this, you give more information to the compiler that it can use to reduce the times when a program unit needs to be invalidated and recompiled. In the above package body, if I did not qualify my reference to n_in inside the SELECT, adding a column named "n_in" will cause the package body to be set to INVALID.

And upon recompilation, well, that reference to "n_in" will now be to the column and not your variable. Not good. By fully qualifying the reference, however, the compiler knows (via fine-grained dependency management, added in 11.1) that there can be no confusion, and adding a column to the employees table could not possibly disrupt the current behavior of the package body.

Thursday, August 10, 2017

No subqueries allowed in materialized view? No problem!

Have you ever run into the following error when trying to create a materialized view?

ORA-22818: subquery expressions not allowed here

Yes, it is true: you cannot have a scalar subquery in the SQL statement used to create your materialized view. Here's an example of what won't work (note: I am not claiming this query makes any sense):

CREATE MATERIALIZED VIEW hr_demo_mv
AS
     SELECT employee_id,
            (SELECT MAX (hire_date)
               FROM employees ce) maxhd
       FROM employees t
/
ORA-22818: subquery expressions not allowed here
22818. 00000 -  "subquery expressions not allowed here"
*Cause:    An attempt was made to use a subquery expression where these
           are not supported.
*Action:   Rewrite the statement without the subquery expression.

Rewrite my query without the subquery expression? But I just spent an hour putting it all together. Works great. Gives me exactly the results I want and need. Rewrite it? ARGH.

Calm yourself. While it is true that you will need to "rewrite the statement" that you provide in your CREATE MATERIALIZED VIEW statement, you will not have to abandon your subqueries and all your hard work.

All you have to do is create a view with the subqueries, and then create your materialized view based on the view:

CREATE VIEW hr_demo_v
AS
     SELECT employee_id,
            (SELECT MAX (hire_date)
               FROM employees ce) maxhd
       FROM employees t
/

View HR_DEMO_V created.

CREATE MATERIALIZED VIEW hr_demo_mv
AS
     SELECT * FROM hr_demo_v
/

Materialized view HR_DEMO_MV created.

I recommend this approach (the materialized view is "nothing more" than a select from a view), even if your materialized view query does not contain a subquery or anything else that would preclude the materialized view from being created.

By taking this approach, you can change the contents of the materialized view with the next refresh by doing nothing more than changing the query (instead of dropping and re-creating the materialized view).

Tuesday, August 8, 2017

Tightening security in your PL/SQL code with 12c new features, part 2

Oracle Database 12c offers several enhancements to improve security in your PL/SQL program units. These features include:
  • Avoid privilege escalation: Use the INHERIT [ANY] PRIVILEGES privilege to make it impossible for a lower-privileged user to take advantage of a higher-privileged user via an invoker rights unit. I covered this topic here.
  • Code-based access control: fine-tune access to database objects inside program units by granting roles to program units (doc), rather than - or in addition to - roles granted to schemas. That's the topic for this post.
Note: Oracle Magazine also offers this content (both blog posts) in a single article here.

Securing your database – and properly restricting access to the data and data structures within your database – ranks at the very top of the "most important things to do" list when building applications.
The best way to avoid unintended access or actions is to apply the "least privilege" principle: give a user the smallest number of (and most narrowly defined) privileges on database objects and the data inside those objects.

Oracle Database has always offered a very robust security mechanism: you can only access objects you own or those to which you were granted access. Within a PL/SQL program unit, you can choose the definer rights model (a user executes your code with your privileges) or the invoker rights model (a user executes your code with their privileges).  But the granularity of this mechanism operates at the schema level, making it difficult to apply the "least privilege" principle.

With Oracle Database 12c, you can now restrict privileges as tightly as you would like, right down to the individual program unit, by granting roles to program units, and not just to schemas. I'll explore this feature for both definer rights and invoker rights program units.

First, with definer rights, suppose that the HR schema was initially granted just two privileges: CREATE SESSION and CREATE PROCEDURE. I could then compile the following procedure in HR:

CREATE OR REPLACE PROCEDURE create_table (
   table_name_in IN VARCHAR2)
   AUTHID DEFINER
IS
BEGIN
   EXECUTE IMMEDIATE
      'CREATE TABLE ' || table_name_in || '(n NUMBER)';
END;

But when I try to create a table using the procedure, I see an error:

CONNECT HR/*****

BEGIN
   create_table ('my_table');
END;
/

ERROR at line 1: ORA-01031: insufficient privileges

Prior to Oracle Database 12c, the only way that HR could use this procedure would be to grant the CREATE TABLE procedure to the schema itself. But this means that any program unit defined in HR could then create a table, which the Chief Security Officer finds unacceptable.

With Oracle Database 12c, however, I can take a much more fine-grained approach, by granting privileges to the procedure itself, and not its owning schema.

Here’s how:

1. Create a role from a schema with the authority to do so, and grant it the CREATE TABLE privilege.

CREATE ROLE create_table_role
/

GRANT CREATE TABLE TO create_table_role
/

2. Grant the role to the procedure. This can be done as SYSDBA. It can also be done from the HR schema, if the role is granted to HR with the admin option. Here’s the grant as SYSDBA:

GRANT create_table_role TO PROCEDURE hr.drop_table
/

To grant from HR, first execute this as SYSDBA:

GRANT create_table_role TO hr WITH ADMIN OPTION
/

ALTER USER hr DEFAULT ROLE ALL EXCEPT create_table_role
/

Then execute the grant from HR:

GRANT create_table_role TO PROCEDURE create_table
/

And now I can execute the procedure and successfully create the table:

BEGIN
   create_table ('my_table');
END;
/

PL/SQL procedure successfully completed.

But if I try to create the table directly, I see the same, earlier privileges error:

CREATE TABLE my_table2 (n NUMBER)
/

ERROR at line 1: ORA-01031: insufficient privileges

The only way to create a table from the HR schema is by calling this one procedure: a very targeted assignment of privileges. Now let's take a look at using code-based access control with an invoker rights module.

With invoker rights, the privileges of the invoking schema are used to determine what the module will be allowed to do. I need to give users the ability to display non-confidential information about employees: namely, they can see employee names and emails, but not salary information.

I can do this by creating a view on top of the EMPLOYEES table and only granting SELECT on the view. But I can also achieve this effect through code based access control, thereby avoiding the need to create a view.

Here's the invoker rights procedure for displaying appropriate employee information, owned by HR, which also owns the employees table.

CREATE OR REPLACE PROCEDURE show_employees (department_id_in IN INTEGER)
   AUTHID CURRENT_USER
AS
BEGIN
   DBMS_OUTPUT.put_line (
      'Employees in Department ' || department_id_in);

   FOR rec IN (SELECT e.last_name, e.email FROM hr.employees e
                WHERE e.department_id = department_id_in
                ORDER BY e.last_name)
   LOOP
      DBMS_OUTPUT.put_line (rec.last_name || ' - ' || rec.email);
   END LOOP;
END;
/

I'll let everyone execute the procedure:

GRANT EXECUTE ON show_employees TO PUBLIC
/

No other schemas have been granted SELECT on employees, so if, for example, a user connected to the SCOTT schema tries to execute this procedure, she will see an error:

BEGIN
   hr.show_employees (10);
END:
/

ERROR at line 1:
ORA-00942: table or view does not exist

Prior to Oracle Database 12c, to get this to work, you would have to do one of the following:
  • Grant SELECT on this table to SCOTT, but that would give SCOTT access to confidential information.
  • Create a view on top of EMPLOYEES that does not include the confidential information, and then grant SELECT on that view to SCOTT.      
With Oracle Database 12c and higher, I can instead create a role that has the SELECT privilege on the EMPLOYEES table, and then assign the role to just that single procedure. Assuming HR has the CREATE ROLE privilege, here are the steps:

CREATE ROLE view_employees_role
/

GRANT SELECT ON employees TO view_employees_role
/

GRANT view_employees_role TO PROCEDURE show_employees
/

BEGIN
   hr.show_employees (10);
END:
/

Employees in Department 10
Whalen – JWHALEN@MY_COMPANY.COM

Now users can access the employee information appropriate to them, but I have not provided any other opportunities to access the employees table. I have, in other words, kept the attack surface (the number of points through which an unauthorized user can try to get at the table) to a minimum.

Monday, July 24, 2017

Document deprecated program units with new pragma (12.2)


Software is constantly evolving: bugs fixed, new features added, and better ways to do things discovered and implemented.

A great example of this dynamic from PL/SQL itself is the UTL_CALL_STACK package. This package was first introduced in Oracle Database 12c Release 1, and it improves upon the functionality already provided by the following functions in the DMBS_UTILITY package: FORMAT_CALL_STACK, FORMAT_ERROR_STACK, and FORMAT_ERROR_BACKTRACE.

The same thing happens in PL/SQL code that is developed by customers. The now-outdated subprograms (or other elements) of one's API cannot be removed immediately; that would break existing code. But everyone would like to make sure that any new code uses the new API.

The new DEPRECATE pragma (compiler directive) in Oracle Database 12.2 will help you accomplish this transition in a smooth, error-free fashion. It provides a formal way to communicate information about deprecated elements [my change ok? SF YES] with a power that ordinary external documentation cannot convey.

You can apply this new pragma to a whole unit, to a subprogram within a unit, at any level of nesting, to any definition, and so on. When a unit is compiled that makes a reference to a deprecated element, a warning is displayed (when you have compile time warnings enabled).

Let's take a look at some examples.

1. Deprecate an entire package.

CREATE PACKAGE pkg AUTHID DEFINER 
AS
   PRAGMA DEPRECATE(pkg);

   PROCEDURE proc;
   FUNCTION func RETURN NUMBER;
END;

2. Deprecate a subprogram in a package. Note the comment added to the pragma. This text will be displayed along with the warning/error information.

CREATE PACKAGE pkg AUTHID DEFINER
AS
  PROCEDURE proc;
  PRAGMA DEPRECATE (
    proc,
    ’pkg.proc deprecated. Use pkg.new_proc instead.’);

  PROCEDURE new_proc;
END;

Let's try using that deprecated procedure, with warnings enabled.

ALTER SESSION SET plsql_warnings = 'enable:all'
/

CREATE OR REPLACE PROCEDURE use_deprecated
   AUTHID DEFINER
IS
BEGIN
   pkg.proc;
END;
/

Procedure USE_DEPRECATED compiled with warnings

PLW-06020: reference to a deprecated entity: PROC declared in unit PKG[4,14]. pkg.proc deprecated. 
   Use pkg.new_proc instead.

Deprecation Warnings

PL/SQL in Oracle Database 12.2 has four new warnings to help you utilize the DEPRECATE pragma:

6019. The entity was deprecated and could be removed in a future release. Do not use the deprecated entity.

6020. The referenced entity was deprecated and could be removed in a future release. Do not use the deprecated entity. Follow the specific instructions in the warning if any are given.

6021. Misplaced pragma. The DEPRECATE pragma should follow immediately after the declaration of the entity that is being deprecated. Place the pragma immediately after the declaration of the entity that is being deprecated.

6022. This entity cannot be deprecated. Deprecation only applies to entities that may be declared in a package or type specification as well as to top-level procedure and function definitions. Remove the pragma.

Try it on LiveSQL!

I know, I know. You'd love to start exploring this and other new features in 12.2, but you don't yet have 12.2 installed at work. No problem!

LiveSQL offers free, 24x7 access to a 12.2 schema. You can execute SQL and PL/SQL code to your heart's content.

You can also run scripts that others, including myself, have uploaded to LiveSQL.

Here's one on the deprecate pragma. Enjoy!

Thursday, July 13, 2017

COUNT Method Works Like COUNT in SQL

You are writing PL/SQL code to provide secure, high performance access to your data and implement business rules. [reference: Why Use PL/SQL?]

Right? Good.

And you use collections (associative arrays, nested tables, arrays) because they offer all sorts of great functionality. [reference: Collections in PL/SQL YouTube playlist]

Right? Good.

So here's a quick reminder about COUNT, one of many methods available for collections (others include DELETE, FIRST, LAST, NEXT, PRIOR, TRIM, EXTEND):

It works pretty much like COUNT in SQL.

If the collection is empty, COUNT returns 0, not NULL.

If you try to "read" an element at an undefined index value, Oracle Database raises NO_DATA_FOUND. Just like a SELECT INTO that identifies no rows.

If you check to see if a collection is empty with a call to COUNT, it doesn't raise NO_DATA_FOUND.

To verify what I've said, and to have a bit of fun while doing it, you can take a quiz on this topic at the Oracle Dev Gym.


Thursday, July 6, 2017

Three tips for getting started right with Oracle Database development

By "Oracle Database development", I mean, more or less, writing SQL and PL/SQL. I assume in this post that you have access to Oracle Database (which you can get via Cloud services, Docker, GitHub and OTN).

A. Use a powerful IDE, designed with database programming in mind.

There are lots of editors out there, and many IDEs that work with Oracle Database. Sure, you could use Notepad, but OMG the productivity loss. You could also use a popular editor like Sublime, and then it get it working with Oracle.

I suggest, however, that you download and install Oracle's own own, free, powerful IDE: SQL Developer.

If you like to complement your graphical IDE with a command line tool (or OMG if you actually prefer a command line tool to a graphical interface), you should also check out the relatively new and generating-lots-of-excitement SQLcl.

B. Enable compile-time warnings and PL/Scope.

The database has tons of useful functionality burned right into it, ready for you to use. For example, when PL/SQL program units are compiled, Oracle can give you feedback (aka, "compile-time warnings) to improve the quality and performance of your code.

In addition, PL/Scope - when enabled - will gather information about your identifiers and (in 12.2) SQL statements. This will allow you to do some very impressive impact analysis of your code.

Most developers are not aware of these features and so leave them turned off. Here's my suggestion for SQL Developer users:

Open up Preferences, type "compile" in the search field. Then change your settings to match these:


In other words:

1. Enable all warnings. 

This way, whenever you compile a program unit, Oracle will give you advice about ways to improve your code.

2. Treat all "severe" warnings as compile-time errors

If the PL/SQL team thinks these warnings are critical in some way, then I want to make my production code is free of such warnings. By setting this caregory to ERROR, I ensure that the code will not compile unless it is "clean". 

3. Tweak your optimization level up to 3 (all the good stuff plus subprogram inlining).

And even more important, take whatever steps are appropriate in your development environment to ensure that production code is compiled at this level of optimization as well. Check out this guidance from the PL/SQL dev team for more details.

4. Turn on PL/Scope.

You can then execute queries against your code to get information regarding naming conventions, sub-optimal code, and opportunities for performance improvements. 

Resources to help you with PL/Scope may be found on LiveSQL and GitHub.
Important Note: These are settings for use in DEVELOPMENT - and they will be applied to all connections made in SQL Developer. When you deploy to production, you should use a script that explicitly sets values for warnings, optimization level (still 3) and PL/Scope (off).
C. Decide RIGHT NOW on logging and instrumentation.

Before you start writing you next program, accept this reality: your code will be full of bugs. You will need to trace execution as well as log those bugs, in order to get your code ready for production and then keep it running smoothly in production.

You need a logging utility for this, and I suggest you use the open-source, widely-used Logger utility available from GitHub.



Wednesday, July 5, 2017

What happens when a package fails to initialize? New behavior as of 12.1!

The best way to build applications on top of Oracle Database is to build lots of APIs (application programmatic interfaces) to your data with PL/SQL packages.

And that means you end up with lots of packages in your application. That's just great!

Now, when a user selects a feature of your application that in turn references an element in a package (invokes a procedure or function, or reads the value of a constant), that package must be instantiated and initialized for that user's session. As described in the documentation:
When a session references a package item, Oracle Database instantiates the package for that session. Every session that references a package has its own instantiation of that package. 
When Oracle Database instantiates a package, it initializes it. Initialization includes whichever of the following are applicable:
  • Assigning initial values to public constants
  • Assigning initial values to public variables whose declarations specify them
  • Executing the initialization part of the package body
Ah, but what happens when any of these steps fail? That, dear reader, is the focus of this post.

Suppose I have a procedure that raises an exception when executed:

CREATE OR REPLACE PROCEDURE always_fails
IS
BEGIN
   RAISE PROGRAM_ERROR;
END;
/

Then no matter how many times I try to run this procedure, it terminates with that same exception:

BEGIN
   always_fails;
END;
/

ORA-06501: PL/SQL: program error
ORA-06512: at "STEVEN.ALWAYS_FAILS", line 4

BEGIN
   always_fails;
END;
/

ORA-06501: PL/SQL: program error
ORA-06512: at "STEVEN.ALWAYS_FAILS", line 4

At which point you must now be saying: "Well, duh, Steven. Of course you are going to see the same exception each time you try to run the procedure."

Exactly. Just so. OK, now let's try it again, with the following package. When the package is initialized, it assigns (or tries to assign) a value of "Lu" to g_name. But that assignment fails, since "Lu" is too big to fit into a VARCHAR2(1) variable.

Thus, the PL/SQL engine raises the VALUE_ERROR exception (ORA-06502).

CREATE OR REPLACE PACKAGE valerr
IS
   FUNCTION little_name RETURN VARCHAR2;
END valerr;
/

Package compiled

CREATE OR REPLACE PACKAGE BODY valerr
IS
   g_name   VARCHAR2 (1) := 'Lu';

   FUNCTION little_name RETURN VARCHAR2
   IS
   BEGIN
      RETURN g_name;
   END little_name;
BEGIN
   DBMS_OUTPUT.put_line ('Before I show you the name...');
EXCEPTION
   WHEN OTHERS
   THEN
      DBMS_OUTPUT.put_line (
         'Trapped the error: ' || DBMS_UTILITY.format_error_stack ());
END valerr;
/

Package body compiled


So what happens when I try to execute the little_name function, after compiling the package?

I see an unhandled exception:

BEGIN
   DBMS_OUTPUT.PUT_LINE ('Name = ' || valerr.little_name);
END;
/

ORA-06502: PL/SQL: numeric or value error: character string buffer too small 
ORA-06512: at "VALERR", line 3

Before going any further, let's make sure you understand why the exception went unhandled. After all, the package body has a "catch-all" exception handler:

EXCEPTION
   WHEN OTHERS
   THEN
      DBMS_OUTPUT.put_line (
         'Trapped the error: ' || DBMS_UTILITY.format_error_stack ());
END valerr;

So why did the exception go unhandled? Because the error occurred in the "declaration section" of the package (not within the initialization section of the package or the executable section of a subprogram of the package). Exception sections only handle errors raised in the executable section of code (see my video for more details).

OK, so now we know:
  1. The package failed to finish initializing.
  2. An exception raised when assigning a default value to a package-level variable or constant cannot be handled within the package.
So if I try to execute this function again, I will see the same error, right? Well, maybe - depending on your version of Oracle Database. Prior to 12.1, you will see this:

BEGIN
   DBMS_OUTPUT.PUT_LINE ('Name = ' || valerr.little_name);
END;
/

Name = 

No exception. Instead, the valerr.little_name function returns a NULL value. Huh?

Yes, I know. That seems counter-intuitive, but here's the thing: prior to Oracle Database 12c Release 1, even if a package failed to initialize, it would be marked as initialized in that session. And any variables or constants that had already successfully been assigned a value would have those values. Which can make it tough to track down the error.

But as of 12.1, when a package fails to initialize, then that package is marked as uninitialized. And any subsequent effort to use that package will grow the same exception. So in 12.1 (and 12.2 and 18.1 and....) you will see:

BEGIN
   DBMS_OUTPUT.PUT_LINE ('Name = ' || valerr.little_name);
END;
/

ORA-06502: PL/SQL: numeric or value error: character string buffer too small  
ORA-06512: at line 2

BEGIN
   DBMS_OUTPUT.PUT_LINE ('Name = ' || valerr.little_name);
END;
/

ORA-06502: PL/SQL: numeric or value error: character string buffer too small  
ORA-06512: at line 2

Conclusion

I expect you will all agree that the 12.1 behavior is preferred to the earlier "Oh, that package has a problem? Not to worry!" approach.

And ideally this change would not result in changed behavior for your application.

As in: hopefully, your testing is good enough so that you would have noticed a package initialization failure.

Finally, if you'd like to test your knowledge on this topic, try our Oracle Dev Gym quiz.