CVS is a relatively basic version control system. For the most part, Subversion has matched or exceeded CVS's feature set where those features continue to apply in Subversion's particular design.
Subversion versions directories as first-class objects, just like files.
Copying and deleting are versioned operations. Renaming is also a versioned operation, albeit with some quirks.
Subversion allows arbitrary metadata ("properties") to be attached to any file or directory. These properties are key/value pairs, and are versioned just like the objects they are attached to. Subversion also provides a way to attach arbitrary key/value properties to a revision (that is, to a committed changeset). These properties are not versioned, since they attach metadata to the version-space itself, but they can be changed at any time.
No part of a commit takes effect until the entire commit has succeeded. Revision numbers are per-commit, not per-file, and commit's log message is attached to its revision, not stored redundantly in all the files affected by that commit.
There is no reason for these operations to be expensive, so they aren't.
Branches and tags are both implemented in terms of an underlying "copy" operation. A copy takes up a small, constant amount of space. Any copy is a tag; and if you start committing on a copy, then it's a branch as well. (This does away with CVS's "branch-point tagging", by removing the distinction that made branch-point tags necessary in the first place.)
Subversion 1.5 introduces merge tracking: automated assistance with managing the flow of changes between lines of development, and with the merging of branches back into their sources. The 1.5 release of merge tracking has basic support for common scenarios; we will be extending the feature in upcoming releases.
Subversion supports (but does not require) locking files so that users can be warned when multiple people try to edit the same file. A file can be marked as requiring a lock before being edited, in which case Subversion will present the file in read-only mode until a lock is acquired.
Unix users can place symbolic links under version control. The links are recreated in Unix working copies, but not in win32 working copies.
Subversion notices when a file is executable, and if that file is placed into version control, its executability will be preserved when it it checked out to other locations. (The mechanism Subversion uses to remember this is simply versioned properties, so executability can be manually edited when necessary, even from a client that does not acknowledge the file's executability, e.g., when having the wrong extension under Microsoft Windows).
Subversion offers a standalone server option using a custom protocol, since not everyone wants to run an Apache HTTPD server. The standalone server can run as an inetd service or in daemon mode, and offers the same level of authentication and authorization functionality as the HTTPD-based server. The standalone server can also be tunnelled over ssh.