Showing posts with label back. Show all posts
Showing posts with label back. Show all posts

Monday, March 26, 2012

latency on transaction repl problem again

Well it only took a few days before the latency bounced back up again. I
don't understand why this is happening. There is another server that has the
same subscription that is operating over a WAN, and it doesn't have the same
problems. Neither do any of the other subscriptions.
Please help if can. Will post the original email below.
Thx,
Kristy[vbcol=seagreen]
I am using transactional replication for a 100GB database that has 2 nosync
subscribers. The pull subscriptions are segmented into like table groups.
They all use a custom profile for the distribution agents that I tweaked
over time to minimize latecy. In the last month I have noticed that 1 of the
subscriptions always has a higher latency in EM then the others. After
looking into several things, we also found out that there was a
hardware/firmware problem with Dell, so we ahve applied that patch and it
has fixed a number of other problems in the DB, but still not the latency
problem on this one subscription. Some of the other subscriptions have much
bigger tables and data changes, so I am at a loss of why this one is causing
me so many problems. I frequently will have to apply a snapshot of this
subscription because it will get so far behind, and then will slwly build
back up again until another snapshot is required.
Can anyone help?
Here are some of the current configs of the custom profile:
BcpBatchsize: 100000
CommitBatchSize: 1000
CommitBatchThreshold: 5000
PollingInterval:1
Thanks,
Kristy
Kristy. Sorry but I write to you for your help, probably not as you expected.
You mentioned in you have 2 nosync subscribers and that is where I am having
problems. Questions for you are: 1) do you create subscription prior to
restoring or another way around? 2) If you create suscription first, what is
the database for the subscription? This is where I am having problems. I use
the instructions of 'How to manually synchronize replication subscriptions by
using backup or restore', article id 320499. Thanks for your helps and sorry
I am not able to help you out.
"Kristy" wrote:

> Well it only took a few days before the latency bounced back up again. I
> don't understand why this is happening. There is another server that has the
> same subscription that is operating over a WAN, and it doesn't have the same
> problems. Neither do any of the other subscriptions.
> Please help if can. Will post the original email below.
> Thx,
> Kristy
> I am using transactional replication for a 100GB database that has 2 nosync
> subscribers. The pull subscriptions are segmented into like table groups.
> They all use a custom profile for the distribution agents that I tweaked
> over time to minimize latecy. In the last month I have noticed that 1 of the
> subscriptions always has a higher latency in EM then the others. After
> looking into several things, we also found out that there was a
> hardware/firmware problem with Dell, so we ahve applied that patch and it
> has fixed a number of other problems in the DB, but still not the latency
> problem on this one subscription. Some of the other subscriptions have much
> bigger tables and data changes, so I am at a loss of why this one is causing
> me so many problems. I frequently will have to apply a snapshot of this
> subscription because it will get so far behind, and then will slwly build
> back up again until another snapshot is required.
> Can anyone help?
> Here are some of the current configs of the custom profile:
> BcpBatchsize: 100000
> CommitBatchSize: 1000
> CommitBatchThreshold: 5000
> PollingInterval:1
> Thanks,
> Kristy
>
>
>
|||Check the history/errors tables for this subscription to try to find out
what is wrong. The tables are in the distribution database and are
msrepl_errors, and msdistribution_history.
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Kristy" <pleasereplyby@.posting.com> wrote in message
news:OFVpZbRTGHA.5104@.TK2MSFTNGP10.phx.gbl...
> Well it only took a few days before the latency bounced back up again. I
> don't understand why this is happening. There is another server that has
> the
> same subscription that is operating over a WAN, and it doesn't have the
> same
> problems. Neither do any of the other subscriptions.
> Please help if can. Will post the original email below.
> Thx,
> Kristy
> I am using transactional replication for a 100GB database that has 2
> nosync
> subscribers. The pull subscriptions are segmented into like table groups.
> They all use a custom profile for the distribution agents that I tweaked
> over time to minimize latecy. In the last month I have noticed that 1 of
> the
> subscriptions always has a higher latency in EM then the others. After
> looking into several things, we also found out that there was a
> hardware/firmware problem with Dell, so we ahve applied that patch and it
> has fixed a number of other problems in the DB, but still not the latency
> problem on this one subscription. Some of the other subscriptions have
> much
> bigger tables and data changes, so I am at a loss of why this one is
> causing
> me so many problems. I frequently will have to apply a snapshot of this
> subscription because it will get so far behind, and then will slwly build
> back up again until another snapshot is required.
> Can anyone help?
> Here are some of the current configs of the custom profile:
> BcpBatchsize: 100000
> CommitBatchSize: 1000
> CommitBatchThreshold: 5000
> PollingInterval:1
> Thanks,
> Kristy
>
>
|||Kathy,
I have to actually create the pull subscriptions twice on the subscriber.
Once for the publisher to see it initially and then when I restore the
database it removes the pull subscriptions from the subscription DB (because
you are overwriting the tables with the new DB that contains the info.) I
then do the pull subscription again on the subscriber and everything works
fine. Publisher does not seem to get confused or think it is a new
subscription.
HTH
Kristy
"Kathy" <Kathy@.discussions.microsoft.com> wrote in message
news:59B2F52B-7ED1-4111-BA18-02C0DA61722E@.microsoft.com...
> Kristy. Sorry but I write to you for your help, probably not as you
expected.
> You mentioned in you have 2 nosync subscribers and that is where I am
having
> problems. Questions for you are: 1) do you create subscription prior to
> restoring or another way around? 2) If you create suscription first, what
is
> the database for the subscription? This is where I am having problems. I
use
> the instructions of 'How to manually synchronize replication subscriptions
by
> using backup or restore', article id 320499. Thanks for your helps and
sorry[vbcol=seagreen]
> I am not able to help you out.
> "Kristy" wrote:
the[vbcol=seagreen]
same[vbcol=seagreen]
nosync[vbcol=seagreen]
groups.[vbcol=seagreen]
the[vbcol=seagreen]
it[vbcol=seagreen]
latency[vbcol=seagreen]
much[vbcol=seagreen]
causing[vbcol=seagreen]
build[vbcol=seagreen]

Monday, March 19, 2012

Last GASP on "Insert row in table with Identity field, and get new Identity back " ?

While I have learned a lot from this thread I am still basically confused about the issues involved.

.I wanted to INSERT a record in a parent table, get the Identity back and use it in a child table. Seems simple.

To my knowledge, mine would be the only process running that would update these tables. I was told that there is no guarantee, because the OLEDB provider could write the second destination row before the first, that the proper parent-child relationship would be generated as expected. It was recommended that I create my own variable in memory to hold the Identity value and use that in my SSIS package.

1. A simple example SSIS .dts example illustrating the approach of using a variable for identity would be helpful.

2. Suppose I actually had two processes updating these tables, running at the same time. Then it seems the "variable" method will also have its problems. Is there a final solution other than locking the tables involved prior to updating them or doing something crazy like using a GUID for the primary key!

3. We have done the type of parent-child inserts I originally described from t-sql for years without any apparent problems. (Maybe we were just lucky.) Is the entire issue simply a t-sql one or does SSIS add a layer of complexity beyond t-sql that needs to be addressed?

TIA,

Barkingdog

SSIS processes data in a different way than T-SQL. The "pipleine" processes differently than batch oriented SQL and differently than cursor based SQL. This is by design of course and has certain benefits and trade-offs, and requires a different way of implementing a solution.

Whether you want to characterize that as an additional "layer of complexity" or not is up to you. I guess if you now have differing options for processing data, that have different advantages and disadvantages, with different costs and benefits, where previously you had only one option , you might consider that more complexity.

I consider it more options - and glad to have 'em...

|||

This post includes examples of how to create surrogate keys using SSIS, effectively the same problem as you are trying to solve: http://sqljunkies.com/WebLog/sqlbi/archive/2005/05/30/15684.aspx

We do not have a packaged solution for highly parallel loads in this version, but we will be releasing a paper on this subject in coming months.

Donald

|||

Thanks,

I will read through the paper.

I am starting to think that, because of parallesim in SSIS, that the sql uniqueidentifier may be a more suitable "primary key" for a table than the traditional Identity value (integer) used in the past.

But I do believe the problem relates to paralellism in SSIS. I don't think this problem would likely occur from a stored proc call that used the "insert a new record, get the identity value, populate the child" approach. (Though I guess it could happen if one had a server farm..... Why is everything so complicated?!)

TIA,

Barkingdog

|||

It's not because of parallelism in SSIS. We have seen this same problem with inserting and immediately retrieving an identity over many years. Unfortunately, everyone thinks they can guarantee the result, and sooner or later they trip up because of some glitch. Unfortunately, because these values are your unique identifiers, such a trip up can be quite serious.

Read:

http://www.aspfaq.com/show.asp?id=2499

http://www.databasejournal.com/features/mssql/article.php/3307541

Donald

|||

>> We have seen this same problem with inserting and immediately retrieving an identity over many years"

The problem I face is that we will be running a single program or SSIS package that is the only program which updates these tables. Nothing else will update them. There are no multiple users, no server farm. I can hear the developers saying that the "problem" mentioned can not happen in that of environemnt. Are they correct or can someone come up with a simple t-sql script that forces the problem to happen in any sql environment?

TIA,

Barkingdog

|||

>The problem I face is that we will be running a single program or SSIS package that is the only program which updates these tables. Nothing else will update them. There are no multiple users, no server farm.

The problem I face is that I have heard this many times from many users - and yet somewhere, somehow, somewhen, the world turns out to be not quite this tidy.

If you think this will work for you, I guess I can only say good luck. But I have an uneasy feeling that in a few months time we'll be reading the "how do I rekey all my child records?" post.

Donald

|||

Thanks Donald. I will follow your sage advice!

I'm just preparing for the bruises\abuse I'm going to get from developers who will think I'm wrong and crazy on this point. I'm sure the URL's you listed will dull their bite.

Barkingdog

|||

Here's what I have learned about this issue so far:

1. This issue has always existed. It is now aggravated by multi-processor systems, server farms, and the parellelism introduced in SSIS.

2. The best thought seems to be to generate one's own counter and use it instead of relying upon an Identity field.

3. While better to use SCOPE_IDENTITY than @.@.Identity or IDENT_CURRENT, even that will not guarantee a correct parent-child linkage. (The problem is if one t-sql operaiton starts before another their is no guarantee it will complete and commit before the other. Forget sequential processing.)

4. I think the only available way to get the proper linkage is to use GUID (from NEWID()) to generate the "identity" key for tables. (But that approach won't be quick for use in JOINS unless I index the GUIDs!)

In summary, I don't think there is a GOOD solution to thie fundamental issue yet. It was mentioned that MicroSoft is working on a best-practices document regarding surrogate keys generation . I can hardly wait for it to be released!

Barkingdog

|||

Actually, I can tell you the technique I have used, and don't recollect ever having a problem with it:

1: ALL rtables must have a _natural_ key defined, and enforced as a unique index.

2: Now after inserting a row into an rtable that uses a system generated surrogate key, you can use the _natural key_ to find the row, therefore it is guaranteed to be the correct row, and retrieve whatever surrogate system generated key was inserted in that row. Obviously there is a performance price to pay to getting the right answer, but typically that's better than a fast wrong answer...

3:now you can use that _guaranteed correct_ surrogate key in further processing as needed.

It has been many years since I have built an rtable that did not enforce uiqueness based on a _natural_ key. It is the only way to go...

Last GASP on "Insert row in table with Identity field, and get new Identity back "

While I have learned a lot from this thread I am still basically confused about the issues involved.

.I wanted to INSERT a record in a parent table, get the Identity back and use it in a child table. Seems simple.

To my knowledge, mine would be the only process running that would update these tables. I was told that there is no guarantee, because the OLEDB provider could write the second destination row before the first, that the proper parent-child relationship would be generated as expected. It was recommended that I create my own variable in memory to hold the Identity value and use that in my SSIS package.

1. A simple example SSIS .dts example illustrating the approach of using a variable for identity would be helpful.

2. Suppose I actually had two processes updating these tables, running at the same time. Then it seems the "variable" method will also have its problems. Is there a final solution other than locking the tables involved prior to updating them or doing something crazy like using a GUID for the primary key!

3. We have done the type of parent-child inserts I originally described from t-sql for years without any apparent problems. (Maybe we were just lucky.) Is the entire issue simply a t-sql one or does SSIS add a layer of complexity beyond t-sql that needs to be addressed?

TIA,

Barkingdog

SSIS processes data in a different way than T-SQL. The "pipleine" processes differently than batch oriented SQL and differently than cursor based SQL. This is by design of course and has certain benefits and trade-offs, and requires a different way of implementing a solution.

Whether you want to characterize that as an additional "layer of complexity" or not is up to you. I guess if you now have differing options for processing data, that have different advantages and disadvantages, with different costs and benefits, where previously you had only one option , you might consider that more complexity.

I consider it more options - and glad to have 'em...

|||

This post includes examples of how to create surrogate keys using SSIS, effectively the same problem as you are trying to solve: http://sqljunkies.com/WebLog/sqlbi/archive/2005/05/30/15684.aspx

We do not have a packaged solution for highly parallel loads in this version, but we will be releasing a paper on this subject in coming months.

Donald

|||

Thanks,

I will read through the paper.

I am starting to think that, because of parallesim in SSIS, that the sql uniqueidentifier may be a more suitable "primary key" for a table than the traditional Identity value (integer) used in the past.

But I do believe the problem relates to paralellism in SSIS. I don't think this problem would likely occur from a stored proc call that used the "insert a new record, get the identity value, populate the child" approach. (Though I guess it could happen if one had a server farm..... Why is everything so complicated?!)

TIA,

Barkingdog

|||

It's not because of parallelism in SSIS. We have seen this same problem with inserting and immediately retrieving an identity over many years. Unfortunately, everyone thinks they can guarantee the result, and sooner or later they trip up because of some glitch. Unfortunately, because these values are your unique identifiers, such a trip up can be quite serious.

Read:

http://www.aspfaq.com/show.asp?id=2499

http://www.databasejournal.com/features/mssql/article.php/3307541

Donald

|||

>> We have seen this same problem with inserting and immediately retrieving an identity over many years"

The problem I face is that we will be running a single program or SSIS package that is the only program which updates these tables. Nothing else will update them. There are no multiple users, no server farm. I can hear the developers saying that the "problem" mentioned can not happen in that of environemnt. Are they correct or can someone come up with a simple t-sql script that forces the problem to happen in any sql environment?

TIA,

Barkingdog

|||

>The problem I face is that we will be running a single program or SSIS package that is the only program which updates these tables. Nothing else will update them. There are no multiple users, no server farm.

The problem I face is that I have heard this many times from many users - and yet somewhere, somehow, somewhen, the world turns out to be not quite this tidy.

If you think this will work for you, I guess I can only say good luck. But I have an uneasy feeling that in a few months time we'll be reading the "how do I rekey all my child records?" post.

Donald

|||

Thanks Donald. I will follow your sage advice!

I'm just preparing for the bruises\abuse I'm going to get from developers who will think I'm wrong and crazy on this point. I'm sure the URL's you listed will dull their bite.

Barkingdog

|||

Here's what I have learned about this issue so far:

1. This issue has always existed. It is now aggravated by multi-processor systems, server farms, and the parellelism introduced in SSIS.

2. The best thought seems to be to generate one's own counter and use it instead of relying upon an Identity field.

3. While better to use SCOPE_IDENTITY than @.@.Identity or IDENT_CURRENT, even that will not guarantee a correct parent-child linkage. (The problem is if one t-sql operaiton starts before another their is no guarantee it will complete and commit before the other. Forget sequential processing.)

4. I think the only available way to get the proper linkage is to use GUID (from NEWID()) to generate the "identity" key for tables. (But that approach won't be quick for use in JOINS unless I index the GUIDs!)

In summary, I don't think there is a GOOD solution to thie fundamental issue yet. It was mentioned that MicroSoft is working on a best-practices document regarding surrogate keys generation . I can hardly wait for it to be released!

Barkingdog

|||

Actually, I can tell you the technique I have used, and don't recollect ever having a problem with it:

1: ALL rtables must have a _natural_ key defined, and enforced as a unique index.

2: Now after inserting a row into an rtable that uses a system generated surrogate key, you can use the _natural key_ to find the row, therefore it is guaranteed to be the correct row, and retrieve whatever surrogate system generated key was inserted in that row. Obviously there is a performance price to pay to getting the right answer, but typically that's better than a fast wrong answer...

3:now you can use that _guaranteed correct_ surrogate key in further processing as needed.

It has been many years since I have built an rtable that did not enforce uiqueness based on a _natural_ key. It is the only way to go...

Friday, March 9, 2012

Large transaction log backups

Hi everyone,
I'm currently running sql 2k with sp3 on win2k with sp4.
I have a maintenance job set up to back up the transaction
logs every four hours every day. I also have it checked
to retain the files for only one day. Afer some intense
nightly jobs, the morning tran log backup can be over 4GB
in size. The problem is not backing up the log to file,
it's the clean-up that's supposed to happen afterwards.
After the tran backup is complete, sql server is supposed
to delete the older tran log backup file from the previous
day, but it always fails to delete the older tran log
backup when it's over 4GB in size. It marks the job as
failed, but the backup finishes so it's not that big a
deal. Is this a known bug or is there something else I'm
missing?
I know I could back up the tran logs more often to reduce
size, but I'm doing log shipping using a custom script and
I'd rather not have 20+ logs to script and ship.
LeonLeon,
First, you might be able to get the size of the log backup down. If the size
is caused by reindexing, perhaps DBCC INDEXDEFRAG will help? Of perhaps
bulk-logged recovery mode? Things you can play with...
As for backup files not being deleted. Perhaps below might help? It is my
"canned" answered for that topic:
Below KB might help:
http://support.microsoft.com/default.aspx?scid=kb;en-us;303292&Product=sql2k
Also, check out below great troubleshooting suggestions from Bill H at MS:
-- Log files don't delete --
This is likely to be either a permissions problem or a sharing violation
problem. The maintenance plan is run as a job, and jobs are run by the
SQLServerAgent service.
Permissions:
1. Determine the startup account for the SQLServerAgent service
(Start|Programs|Administrative tools|Services|SQLServerAgent|Startup). This
account is the security context for jobs, and thus the maintenance plan.
2. If SQLServerAgent is started using LocalSystem (as opposed to a domain
account) then skip step 3.
3. On that box, log onto NT as that account. Using Explorer, attempt to
delete an expired backup. If that succeeds then go to Sharing Violation
section.
4. Log onto NT with an account that is an administrator and use Explorer to
look at the Properties|Security of the folder (where the backups reside)
and ensure the SQLServerAgent startup account has Full Control. If the
SQLServerAgent startup account is LocalSystem, then the account to consider
is SYSTEM.
5. In NT, if an account is a member of an NT group, and if that group has
Access is Denied, then that account will have Access is Denied, even if
that account is also a member of the Administrators group. Thus you may
need to check group permissions (if the Startup Account is a member of a
group).
6. Keep in mind that permissions (by default) are inherited from a parent
folder. Thus, if the backups are stored in C:\bak, and if someone had
denied permission to the SQLServerAgent startup account for C:\, then
C:\bak will inherit access is denied.
Sharing violation:
This is likely to be rooted in a timing issue, with the most likely cause
being another scheduled process (such as NT Backup or Anti-Virus software)
having the backup file open at the time when the SQLServerAgent (i.e., the
maintenance plan job) tried to delete it.
1. Download filemon and handle from www.sysinternals.com.
2. I am not sure whether filemon can be scheduled, or you might be able to
use NT scheduling services to start filemon just before the maintenance
plan job is started, but the filemon log can become very large, so it would
be best to start it some short time before the maintenance plan starts.
3. Inspect the filemon log for another process that has that backup file
open (if your lucky enough to have started filemon before this other
process grabs the backup folder), and inspect the log for the results when
the SQLServerAgent agent attempts to open that same file.
4. Schedule the job or that other process to do their work at different
times.
5. You can use the handle utility if you are around at the time when the
job is scheduled to run.
If the backup files are going to a \\share or a mapped drive (as opposed to
local drive), then you will need to modify the above (with respect to where
the tests and utilities are run).
Finally, inspection of the maintenance plan's history report might be
useful.
Thanks,
Bill Hollinshead
Microsoft, SQL Server
Tibor Karaszi, SQL Server MVP
Archive at:
http://groups.google.com/groups?oi=djq&as_ugroup=microsoft.public.sqlserver
"Leon" <anonymous@.discussions.microsoft.com> wrote in message
news:1320001c3f6f9$e945c820$a601280a@.phx.gbl...
> Hi everyone,
> I'm currently running sql 2k with sp3 on win2k with sp4.
> I have a maintenance job set up to back up the transaction
> logs every four hours every day. I also have it checked
> to retain the files for only one day. Afer some intense
> nightly jobs, the morning tran log backup can be over 4GB
> in size. The problem is not backing up the log to file,
> it's the clean-up that's supposed to happen afterwards.
> After the tran backup is complete, sql server is supposed
> to delete the older tran log backup file from the previous
> day, but it always fails to delete the older tran log
> backup when it's over 4GB in size. It marks the job as
> failed, but the backup finishes so it's not that big a
> deal. Is this a known bug or is there something else I'm
> missing?
> I know I could back up the tran logs more often to reduce
> size, but I'm doing log shipping using a custom script and
> I'd rather not have 20+ logs to script and ship.
> Leon|||Tibor,
Thanks a bunch for the article and the suggestions from
Bill.
Leon
>--Original Message--
>Leon,
>First, you might be able to get the size of the log
backup down. If the size
>is caused by reindexing, perhaps DBCC INDEXDEFRAG will
help? Of perhaps
>bulk-logged recovery mode? Things you can play with...
>As for backup files not being deleted. Perhaps below
might help? It is my
>"canned" answered for that topic:
>Below KB might help:
>http://support.microsoft.com/default.aspx?scid=kb;en-
us;303292&Product=sql2k
>
>Also, check out below great troubleshooting suggestions
from Bill H at MS:
>
>-- Log files don't delete --
>This is likely to be either a permissions problem or a
sharing violation
>problem. The maintenance plan is run as a job, and jobs
are run by the
>SQLServerAgent service.
>Permissions:
>1. Determine the startup account for the SQLServerAgent
service
>(Start|Programs|Administrative
tools|Services|SQLServerAgent|Startup). This
>account is the security context for jobs, and thus the
maintenance plan.
>2. If SQLServerAgent is started using LocalSystem (as
opposed to a domain
>account) then skip step 3.
>3. On that box, log onto NT as that account. Using
Explorer, attempt to
>delete an expired backup. If that succeeds then go to
Sharing Violation
>section.
>4. Log onto NT with an account that is an administrator
and use Explorer to
>look at the Properties|Security of the folder (where the
backups reside)
>and ensure the SQLServerAgent startup account has Full
Control. If the
>SQLServerAgent startup account is LocalSystem, then the
account to consider
>is SYSTEM.
>5. In NT, if an account is a member of an NT group, and
if that group has
>Access is Denied, then that account will have Access is
Denied, even if
>that account is also a member of the Administrators
group. Thus you may
>need to check group permissions (if the Startup Account
is a member of a
>group).
>6. Keep in mind that permissions (by default) are
inherited from a parent
>folder. Thus, if the backups are stored in C:\bak, and if
someone had
>denied permission to the SQLServerAgent startup account
for C:\, then
>C:\bak will inherit access is denied.
>Sharing violation:
>This is likely to be rooted in a timing issue, with the
most likely cause
>being another scheduled process (such as NT Backup or
Anti-Virus software)
>having the backup file open at the time when the
SQLServerAgent (i.e., the
>maintenance plan job) tried to delete it.
>1. Download filemon and handle from www.sysinternals.com.
>2. I am not sure whether filemon can be scheduled, or you
might be able to
>use NT scheduling services to start filemon just before
the maintenance
>plan job is started, but the filemon log can become very
large, so it would
>be best to start it some short time before the
maintenance plan starts.
>3. Inspect the filemon log for another process that has
that backup file
>open (if your lucky enough to have started filemon before
this other
>process grabs the backup folder), and inspect the log for
the results when
>the SQLServerAgent agent attempts to open that same file.
>4. Schedule the job or that other process to do their
work at different
>times.
>5. You can use the handle utility if you are around at
the time when the
>job is scheduled to run.
>If the backup files are going to a \\share or a mapped
drive (as opposed to
>local drive), then you will need to modify the above
(with respect to where
>the tests and utilities are run).
>Finally, inspection of the maintenance plan's history
report might be
>useful.
>Thanks,
>Bill Hollinshead
>Microsoft, SQL Server
>
>
>--
>Tibor Karaszi, SQL Server MVP
>Archive at:
>http://groups.google.com/groups?
oi=djq&as_ugroup=microsoft.public.sqlserver
>
>"Leon" <anonymous@.discussions.microsoft.com> wrote in
message
>news:1320001c3f6f9$e945c820$a601280a@.phx.gbl...
>> Hi everyone,
>> I'm currently running sql 2k with sp3 on win2k with sp4.
>> I have a maintenance job set up to back up the
transaction
>> logs every four hours every day. I also have it checked
>> to retain the files for only one day. Afer some intense
>> nightly jobs, the morning tran log backup can be over
4GB
>> in size. The problem is not backing up the log to file,
>> it's the clean-up that's supposed to happen afterwards.
>> After the tran backup is complete, sql server is
supposed
>> to delete the older tran log backup file from the
previous
>> day, but it always fails to delete the older tran log
>> backup when it's over 4GB in size. It marks the job as
>> failed, but the backup finishes so it's not that big a
>> deal. Is this a known bug or is there something else
I'm
>> missing?
>> I know I could back up the tran logs more often to
reduce
>> size, but I'm doing log shipping using a custom script
and
>> I'd rather not have 20+ logs to script and ship.
>> Leon
>
>.
>

Wednesday, March 7, 2012

Large transaction log backups

Hi everyone,
I'm currently running sql 2k with sp3 on win2k with sp4.
I have a maintenance job set up to back up the transaction
logs every four hours every day. I also have it checked
to retain the files for only one day. Afer some intense
nightly jobs, the morning tran log backup can be over 4GB
in size. The problem is not backing up the log to file,
it's the clean-up that's supposed to happen afterwards.
After the tran backup is complete, sql server is supposed
to delete the older tran log backup file from the previous
day, but it always fails to delete the older tran log
backup when it's over 4GB in size. It marks the job as
failed, but the backup finishes so it's not that big a
deal. Is this a known bug or is there something else I'm
missing?
I know I could back up the tran logs more often to reduce
size, but I'm doing log shipping using a custom script and
I'd rather not have 20+ logs to script and ship.
LeonLeon,
First, you might be able to get the size of the log backup down. If the size
is caused by reindexing, perhaps DBCC INDEXDEFRAG will help? Of perhaps
bulk-logged recovery mode? Things you can play with...
As for backup files not being deleted. Perhaps below might help? It is my
"canned" answered for that topic:
Below KB might help:
http://support.microsoft.com/defaul...2&Product=sql2k
Also, check out below great troubleshooting suggestions from Bill H at MS:
-- Log files don't delete --
This is likely to be either a permissions problem or a sharing violation
problem. The maintenance plan is run as a job, and jobs are run by the
SQLServerAgent service.
Permissions:
1. Determine the startup account for the SQLServerAgent service
(Start|Programs|Administrative tools|Services|SQLServerAgent|Startup). This
account is the security context for jobs, and thus the maintenance plan.
2. If SQLServerAgent is started using LocalSystem (as opposed to a domain
account) then skip step 3.
3. On that box, log onto NT as that account. Using Explorer, attempt to
delete an expired backup. If that succeeds then go to Sharing Violation
section.
4. Log onto NT with an account that is an administrator and use Explorer to
look at the Properties|Security of the folder (where the backups reside)
and ensure the SQLServerAgent startup account has Full Control. If the
SQLServerAgent startup account is LocalSystem, then the account to consider
is SYSTEM.
5. In NT, if an account is a member of an NT group, and if that group has
Access is Denied, then that account will have Access is Denied, even if
that account is also a member of the Administrators group. Thus you may
need to check group permissions (if the Startup Account is a member of a
group).
6. Keep in mind that permissions (by default) are inherited from a parent
folder. Thus, if the backups are stored in C:\bak, and if someone had
denied permission to the SQLServerAgent startup account for C:\, then
C:\bak will inherit access is denied.
Sharing violation:
This is likely to be rooted in a timing issue, with the most likely cause
being another scheduled process (such as NT Backup or Anti-Virus software)
having the backup file open at the time when the SQLServerAgent (i.e., the
maintenance plan job) tried to delete it.
1. Download filemon and handle from www.sysinternals.com.
2. I am not sure whether filemon can be scheduled, or you might be able to
use NT scheduling services to start filemon just before the maintenance
plan job is started, but the filemon log can become very large, so it would
be best to start it some short time before the maintenance plan starts.
3. Inspect the filemon log for another process that has that backup file
open (if your lucky enough to have started filemon before this other
process grabs the backup folder), and inspect the log for the results when
the SQLServerAgent agent attempts to open that same file.
4. Schedule the job or that other process to do their work at different
times.
5. You can use the handle utility if you are around at the time when the
job is scheduled to run.
If the backup files are going to a \\share or a mapped drive (as opposed to
local drive), then you will need to modify the above (with respect to where
the tests and utilities are run).
Finally, inspection of the maintenance plan's history report might be
useful.
Thanks,
Bill Hollinshead
Microsoft, SQL Server
Tibor Karaszi, SQL Server MVP
Archive at:
http://groups.google.com/groups?oi=...ublic.sqlserver
"Leon" <anonymous@.discussions.microsoft.com> wrote in message
news:1320001c3f6f9$e945c820$a601280a@.phx
.gbl...
> Hi everyone,
> I'm currently running sql 2k with sp3 on win2k with sp4.
> I have a maintenance job set up to back up the transaction
> logs every four hours every day. I also have it checked
> to retain the files for only one day. Afer some intense
> nightly jobs, the morning tran log backup can be over 4GB
> in size. The problem is not backing up the log to file,
> it's the clean-up that's supposed to happen afterwards.
> After the tran backup is complete, sql server is supposed
> to delete the older tran log backup file from the previous
> day, but it always fails to delete the older tran log
> backup when it's over 4GB in size. It marks the job as
> failed, but the backup finishes so it's not that big a
> deal. Is this a known bug or is there something else I'm
> missing?
> I know I could back up the tran logs more often to reduce
> size, but I'm doing log shipping using a custom script and
> I'd rather not have 20+ logs to script and ship.
> Leon|||Tibor,
Thanks a bunch for the article and the suggestions from
Bill.
Leon

>--Original Message--
>Leon,
>First, you might be able to get the size of the log
backup down. If the size
>is caused by reindexing, perhaps DBCC INDEXDEFRAG will
help? Of perhaps
>bulk-logged recovery mode? Things you can play with...
>As for backup files not being deleted. Perhaps below
might help? It is my
>"canned" answered for that topic:
>Below KB might help:
>http://support.microsoft.com/default.aspx?scid=kb;en-
us;303292&Product=sql2k
>
>Also, check out below great troubleshooting suggestions
from Bill H at MS:
>
>-- Log files don't delete --
>This is likely to be either a permissions problem or a
sharing violation
>problem. The maintenance plan is run as a job, and jobs
are run by the
>SQLServerAgent service.
>Permissions:
>1. Determine the startup account for the SQLServerAgent
service
>(Start|Programs|Administrative
tools|Services|SQLServerAgent|Startup). This
>account is the security context for jobs, and thus the
maintenance plan.
>2. If SQLServerAgent is started using LocalSystem (as
opposed to a domain
>account) then skip step 3.
>3. On that box, log onto NT as that account. Using
Explorer, attempt to
>delete an expired backup. If that succeeds then go to
Sharing Violation
>section.
>4. Log onto NT with an account that is an administrator
and use Explorer to
>look at the Properties|Security of the folder (where the
backups reside)
>and ensure the SQLServerAgent startup account has Full
Control. If the
>SQLServerAgent startup account is LocalSystem, then the
account to consider
>is SYSTEM.
>5. In NT, if an account is a member of an NT group, and
if that group has
>Access is Denied, then that account will have Access is
Denied, even if
>that account is also a member of the Administrators
group. Thus you may
>need to check group permissions (if the Startup Account
is a member of a
>group).
>6. Keep in mind that permissions (by default) are
inherited from a parent
>folder. Thus, if the backups are stored in C:\bak, and if
someone had
>denied permission to the SQLServerAgent startup account
for C:\, then
>C:\bak will inherit access is denied.
>Sharing violation:
>This is likely to be rooted in a timing issue, with the
most likely cause
>being another scheduled process (such as NT Backup or
Anti-Virus software)
>having the backup file open at the time when the
SQLServerAgent (i.e., the
>maintenance plan job) tried to delete it.
>1. Download filemon and handle from www.sysinternals.com.
>2. I am not sure whether filemon can be scheduled, or you
might be able to
>use NT scheduling services to start filemon just before
the maintenance
>plan job is started, but the filemon log can become very
large, so it would
>be best to start it some short time before the
maintenance plan starts.
>3. Inspect the filemon log for another process that has
that backup file
>open (if your lucky enough to have started filemon before
this other
>process grabs the backup folder), and inspect the log for
the results when
>the SQLServerAgent agent attempts to open that same file.
>4. Schedule the job or that other process to do their
work at different
>times.
>5. You can use the handle utility if you are around at
the time when the
>job is scheduled to run.
>If the backup files are going to a \\share or a mapped
drive (as opposed to
>local drive), then you will need to modify the above
(with respect to where
>the tests and utilities are run).
>Finally, inspection of the maintenance plan's history
report might be
>useful.
>Thanks,
>Bill Hollinshead
>Microsoft, SQL Server
>
>
>--
>Tibor Karaszi, SQL Server MVP
>Archive at:
>http://groups.google.com/groups?
oi=djq&as_ugroup=microsoft.public.sqlserver
>
>"Leon" <anonymous@.discussions.microsoft.com> wrote in
message
> news:1320001c3f6f9$e945c820$a601280a@.phx
.gbl...
transaction
4GB
supposed
previous
I'm
reduce
and
>
>.
>

Friday, February 24, 2012

Large scale changes of data in SQL Server 2005

Hi
We are using a Helpdesk system which uses SQL Server 2005 as the back
end. Our office is the group HQ for all our subsidaries and affiliates,
so we tend to get a lot of IT support calls from different
organisations.
When HD agents log the call, a field exists to identify which
organisation this is, e.g. Company A, Company B and so on.
Company A has decided to change their name to Company Z. This needs to
be reflected in the software not just from this point on, but also
historically, i.e. even a called in 2005 by an employee of Company A
should now read as a call logged by an employee of Company Z.
What is the best method to change this in the database? Luckily,
Company A has been one of the smaller companies, so the total amount of
calls by them that exist is about 40.
Many thanks in advance.
SJHi
I hope you do have Company table where you store the names of the companies
and their ID
UPDATE Company SET [Name]='Company Z' WHERE [Name]='Company B'
If your database design is proper you don't need change anything because
the table is refernced by ID anot by NAME
<smokejo@.googlemail.com> wrote in message
news:1163971948.529460.64020@.f16g2000cwb.googlegroups.com...
> Hi
> We are using a Helpdesk system which uses SQL Server 2005 as the back
> end. Our office is the group HQ for all our subsidaries and affiliates,
> so we tend to get a lot of IT support calls from different
> organisations.
> When HD agents log the call, a field exists to identify which
> organisation this is, e.g. Company A, Company B and so on.
> Company A has decided to change their name to Company Z. This needs to
> be reflected in the software not just from this point on, but also
> historically, i.e. even a called in 2005 by an employee of Company A
> should now read as a call logged by an employee of Company Z.
> What is the best method to change this in the database? Luckily,
> Company A has been one of the smaller companies, so the total amount of
> calls by them that exist is about 40.
> Many thanks in advance.
> SJ
>|||Uri Dimant wrote:
> Hi
> I hope you do have Company table where you store the names of the companies
> and their ID
> UPDATE Company SET [Name]='Company Z' WHERE [Name]='Company B'
> If your database design is proper you don't need change anything because
> the table is refernced by ID anot by NAME
>
We have a Company table and within it columns for CompanyID and
CompanyName.
I have a couple of questions though...
i) How is it possible to 'see' what values there are for the Company
Name?
ii) Where do I run the command that you mentioned?
iii) Is it possible to actually see all the data in the database and,
if so, how can I do that? For instance, in MS Access you can view the
database and all the info in it...
Sorry, I'm not really an SQL man, so have no idea!
Thanks so much in advance...|||> i) How is it possible to 'see' what values there are for the Company
> Name?
SELECT * FROM Table WHERE NOT EXISTS (SELECT * FROM Company C WHERE
C.CompanyID=Table.CompanyID)
> ii) Where do I run the command that you mentioned?
Query Analyzer
> iii) Is it possible to actually see all the data in the database and,
What did you mean?
> if so, how can I do that? For instance, in MS Access you can view the
> database and all the info in it...
Enterprise Manager?
<smokejo@.googlemail.com> wrote in message
news:1164110500.619811.76700@.j44g2000cwa.googlegroups.com...
> Uri Dimant wrote:
>> Hi
>> I hope you do have Company table where you store the names of the
>> companies
>> and their ID
>> UPDATE Company SET [Name]='Company Z' WHERE [Name]='Company B'
>> If your database design is proper you don't need change anything because
>> the table is refernced by ID anot by NAME
>>
> We have a Company table and within it columns for CompanyID and
> CompanyName.
> I have a couple of questions though...
> i) How is it possible to 'see' what values there are for the Company
> Name?
> ii) Where do I run the command that you mentioned?
> iii) Is it possible to actually see all the data in the database and,
> if so, how can I do that? For instance, in MS Access you can view the
> database and all the info in it...
> Sorry, I'm not really an SQL man, so have no idea!
> Thanks so much in advance...
>