|
web
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
System.IO.FileSystemWatcher is missing alot of files.I have a Windows Form application that monitors a directory using the
System.IO.FileSystemWatcher. On the event created I run a sub that sends an xml file over http to a service. When I copy 7000 files in the directory to be monitored only 123 are sent to the service. What might be going wrong? More info: The sending of the XML is more then just the http post. I also have to do some stuff with xml first before sending it.
Show quote
Hide quote
"Philip Wagenaar" <philip.wagenaar@online.nospam> wrote in message news:1A09EA20-078F-4A5A-AAB7-B909F676D3F5@microsoft.com... It's possible that you are missing events because it takes longer to process the files than it takes to generate the events and thus> I have a Windows Form application that monitors a directory using the > System.IO.FileSystemWatcher. > > On the event created I run a sub that sends an xml file over http to a > service. > > When I copy 7000 files in the directory to be monitored only 123 are sent to > the service. > > What might be going wrong? > > More info: > > The sending of the XML is more then just the http post. I also have to do > some stuff with xml first before sending it. some get lost. I had a similar situation using the FileSystemWatcher and I solved it by placing the event info into a FIFO and had a separate worker thread unloading the FIFO and processing the events. This completely eliminated the problem. Perhaps this approach would be appropriate here as well. -- Al Reid You mean working with threads?
Is it possible to post a sample? Show quoteHide quote "Al Reid" wrote: > "Philip Wagenaar" <philip.wagenaar@online.nospam> wrote in message news:1A09EA20-078F-4A5A-AAB7-B909F676D3F5@microsoft.com... > > I have a Windows Form application that monitors a directory using the > > System.IO.FileSystemWatcher. > > > > On the event created I run a sub that sends an xml file over http to a > > service. > > > > When I copy 7000 files in the directory to be monitored only 123 are sent to > > the service. > > > > What might be going wrong? > > > > More info: > > > > The sending of the XML is more then just the http post. I also have to do > > some stuff with xml first before sending it. > > It's possible that you are missing events because it takes longer to process the files than it takes to generate the events and thus > some get lost. I had a similar situation using the FileSystemWatcher and I solved it by placing the event info into a FIFO and had > a separate worker thread unloading the FIFO and processing the events. This completely eliminated the problem. Perhaps this > approach would be appropriate here as well. > > > -- > Al Reid > > > "Philip Wagenaar" <philip.wagenaar@online.nospam> wrote in message news:5FCD0C34-9C97-41C5-AC95-015D2A92A4BD@microsoft.com... Yes, I mean working with threads. If I get a chance, I'll see if I can't skinny down my working code and post the basics.> You mean working with threads? > > Is it possible to post a sample? > > "Al Reid" wrote: > -- Show quoteHide quoteAl Reid > > > > It's possible that you are missing events because it takes longer to process the files than it takes to generate the events and thus > > some get lost. I had a similar situation using the FileSystemWatcher and I solved it by placing the event info into a FIFO and had > > a separate worker thread unloading the FIFO and processing the events. This completely eliminated the problem. Perhaps this > > approach would be appropriate here as well. > > > > > > -- > > Al Reid > > > > > > "Philip Wagenaar" <philip.wagenaar@online.nospam> wrote in message news:5FCD0C34-9C97-41C5-AC95-015D2A92A4BD@microsoft.com... I found my original proof of concept code. It was implemented as a Windows Service. Most of the concepts were obtained from Google> You mean working with threads? > > Is it possible to post a sample? > searches, etc. ================================================= Imports System Imports System.Threading Module General Public goFIFO As New Collections.Queue Public gblnStopThreads As Boolean Public gstrSourceDir As String Public gstrDestinationDir As String Public goLogger As New CLog Public goEventlog As EventLog Public gintIdle As Integer Public gintInterval As Integer End Module ----------------------------- Imports System.Threading Imports System.io Public Class WorkerThread Public Shared Sub ProcessFiles() Dim e As Object Dim strFilePath As String Dim strDestDir As String Do Until gblnStopThreads And (goFIFO.Count = 0) Try If goFIFO.Count > 0 Then e = goFIFO.Dequeue If TypeOf e Is RenamedEventArgs Then Dim strOldName As String = CType(e, RenamedEventArgs).OldFullPath.Replace(gstrSourceDir, gstrDestinationDir) strDestDir = CType(e, RenamedEventArgs).FullPath.Replace(gstrSourceDir, gstrDestinationDir) Rename(strOldName, strDestDir) ElseIf TypeOf e Is FileSystemEventArgs Then strFilePath = CType(e, FileSystemEventArgs).FullPath strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) If (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then If Not Directory.Exists(strDestDir) Then strFilePath = strFilePath strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) Directory.CreateDirectory(strDestDir) End If ElseIf New FileInfo(strDestDir).Directory.Exists = False Then strFilePath = strFilePath strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) Directory.CreateDirectory(Path.GetDirectoryName(strDestDir)) End If If Not (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then FileCopy(strFilePath, strDestDir) End If End If Thread.Sleep(gintInterval) Else Thread.Sleep(gintIdle) End If Catch ex As Exception goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error) End Try Loop End Sub End Class ----------------------------- Imports System.ServiceProcess Imports System Imports System.Threading Imports System.io Imports Microsoft.Win32 Public Class MirrorService Inherits System.ServiceProcess.ServiceBase Private WithEvents moWatcher As FileSystemWatcher Protected Overrides Sub OnStart(ByVal args() As String) Me.AutoLog = True goEventlog = Me.EventLog() Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High gstrSourceDir = "C:\WatcherTest\Source" gstrDestinationDir = "F:\WatcherTest\Destination" gintIdle = 5000 gintInterval = 100 Dim oWorker As New WorkerThread Dim oWorkerThread As New Thread(New ThreadStart(AddressOf oWorker.ProcessFiles)) ' Start the thread. oWorkerThread.Start() moWatcher = New FileSystemWatcher moWatcher.InternalBufferSize = 16384 moWatcher.Path = gstrSourceDir moWatcher.IncludeSubdirectories = True moWatcher.EnableRaisingEvents = True End Sub Protected Overrides Sub OnStop() gblnStopThreads = True End Sub Private Sub moWatcher_CreatedOrChanged(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles moWatcher.Changed, moWatcher.Created Try If e.ChangeType <> WatcherChangeTypes.Deleted Then goFIFO.Enqueue(e) End If Catch ex As Exception goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error) End Try End Sub Private Sub moWatcher_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles moWatcher.Renamed goFIFO.Enqueue(CType(e, RenamedEventArgs)) End Sub End Class =================================================== Sorry that the cut and paste lost the formatting. I hope this helps. == Al Reid Thanks! This really helped me alot!
Show quoteHide quote "Al Reid" wrote: > "Philip Wagenaar" <philip.wagenaar@online.nospam> wrote in message news:5FCD0C34-9C97-41C5-AC95-015D2A92A4BD@microsoft.com... > > You mean working with threads? > > > > Is it possible to post a sample? > > > > I found my original proof of concept code. It was implemented as a Windows Service. Most of the concepts were obtained from Google > searches, etc. > > ================================================= > > Imports System > > Imports System.Threading > > Module General > > Public goFIFO As New Collections.Queue > > Public gblnStopThreads As Boolean > > Public gstrSourceDir As String > > Public gstrDestinationDir As String > > Public goLogger As New CLog > > Public goEventlog As EventLog > > Public gintIdle As Integer > > Public gintInterval As Integer > > End Module > > ----------------------------- > > Imports System.Threading > > Imports System.io > > Public Class WorkerThread > > Public Shared Sub ProcessFiles() > > Dim e As Object > > Dim strFilePath As String > > Dim strDestDir As String > > Do Until gblnStopThreads And (goFIFO.Count = 0) > > Try > > If goFIFO.Count > 0 Then > > e = goFIFO.Dequeue > > If TypeOf e Is RenamedEventArgs Then > > Dim strOldName As String = CType(e, RenamedEventArgs).OldFullPath.Replace(gstrSourceDir, gstrDestinationDir) > > strDestDir = CType(e, RenamedEventArgs).FullPath.Replace(gstrSourceDir, gstrDestinationDir) > > Rename(strOldName, strDestDir) > > ElseIf TypeOf e Is FileSystemEventArgs Then > > strFilePath = CType(e, FileSystemEventArgs).FullPath > > strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) > > If (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then > > If Not Directory.Exists(strDestDir) Then > > strFilePath = strFilePath > > strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) > > Directory.CreateDirectory(strDestDir) > > End If > > ElseIf New FileInfo(strDestDir).Directory.Exists = False Then > > strFilePath = strFilePath > > strDestDir = strFilePath.Replace(gstrSourceDir, gstrDestinationDir) > > Directory.CreateDirectory(Path.GetDirectoryName(strDestDir)) > > End If > > If Not (GetAttr(strFilePath) And FileAttribute.Directory) = FileAttribute.Directory Then > > FileCopy(strFilePath, strDestDir) > > End If > > End If > > Thread.Sleep(gintInterval) > > Else > > Thread.Sleep(gintIdle) > > End If > > Catch ex As Exception > > goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error) > > End Try > > Loop > > End Sub > > End Class > > > > ----------------------------- > > > > Imports System.ServiceProcess > > Imports System > > Imports System.Threading > > Imports System.io > > Imports Microsoft.Win32 > > Public Class MirrorService > > Inherits System.ServiceProcess.ServiceBase > > Private WithEvents moWatcher As FileSystemWatcher > > Protected Overrides Sub OnStart(ByVal args() As String) > > Me.AutoLog = True > > goEventlog = Me.EventLog() > > Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High > > gstrSourceDir = "C:\WatcherTest\Source" > > gstrDestinationDir = "F:\WatcherTest\Destination" > > gintIdle = 5000 > > gintInterval = 100 > > Dim oWorker As New WorkerThread > > Dim oWorkerThread As New Thread(New ThreadStart(AddressOf oWorker.ProcessFiles)) > > ' Start the thread. > > oWorkerThread.Start() > > moWatcher = New FileSystemWatcher > > moWatcher.InternalBufferSize = 16384 > > moWatcher.Path = gstrSourceDir > > moWatcher.IncludeSubdirectories = True > > moWatcher.EnableRaisingEvents = True > > End Sub > > Protected Overrides Sub OnStop() > > gblnStopThreads = True > > End Sub > > Private Sub moWatcher_CreatedOrChanged(ByVal sender As Object, ByVal e As System.IO.FileSystemEventArgs) Handles moWatcher.Changed, > moWatcher.Created > > Try > > If e.ChangeType <> WatcherChangeTypes.Deleted Then > > goFIFO.Enqueue(e) > > End If > > Catch ex As Exception > > goEventlog.WriteEntry(ex.Message, EventLogEntryType.Error) > > End Try > > End Sub > > Private Sub moWatcher_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles moWatcher.Renamed > > goFIFO.Enqueue(CType(e, RenamedEventArgs)) > > End Sub > > End Class > > =================================================== > > > > Sorry that the cut and paste lost the formatting. I hope this helps. > > == > > Al Reid > > >
ADO.net + MS Access = performance issues
VB.NET Structures and Union Help. how to group similar events? 2nd Post: Problem adding events to controls created at run-time KeyUp + KeyDown Event Handler Calculate elapsed time Multiple threads using a shared printer resource Problems with ByRef parameters Copying files across network continue debugging without interrupting |
|||||||||||||||||||||||